{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![MLU Logo](../data/MLU_Logo.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <a name=\"0\">Machine Learning Accelerator - Tabular Data - Lecture 3</a>\n",
    "\n",
    "\n",
    "## Neural Network Model\n",
    "\n",
    "In this notebook, we build, train, validate and test a Neural Network with PyTorch to predict the __Outcome Type__ field of our review dataset.\n",
    "\n",
    "\n",
    "1. <a href=\"#1\">Read the dataset</a>\n",
    "2. <a href=\"#2\">Exploratory Data Analysis</a>\n",
    "3. <a href=\"#3\">Select features to build the model</a>\n",
    "4. <a href=\"#4\">Training, validation, and test datasets</a>\n",
    "5. <a href=\"#5\">Data processing with Pipeline and ColumnTransformer</a>\n",
    "6. <a href=\"#6\">Neural network training and validation</a>\n",
    "7. <a href=\"#7\">Test the neural network</a>\n",
    "8. <a href=\"#8\">Improvement ideas</a>\n",
    "\n",
    "__Austin Animal Center Dataset__:\n",
    "\n",
    "In this exercise, we are working with pet adoption data from __Austin Animal Center__. We have two datasets that cover intake and outcome of animals. Intake data is available from [here](https://data.austintexas.gov/Health-and-Community-Services/Austin-Animal-Center-Intakes/wter-evkm) and outcome is from [here](https://data.austintexas.gov/Health-and-Community-Services/Austin-Animal-Center-Outcomes/9t4d-g238). \n",
    "\n",
    "In order to work with a single table, we joined the intake and outcome tables using the \"Animal ID\" column and created a single __review.csv__ file. We also didn't consider animals with multiple entries to the facility to keep our dataset simple. If you want to see the original datasets and the merged data with multiple entries, they are available under data/review folder: Austin_Animal_Center_Intakes.csv, Austin_Animal_Center_Outcomes.csv and Austin_Animal_Center_Intakes_Outcomes.csv.\n",
    "\n",
    "__Dataset schema:__ \n",
    "- __Pet ID__ - Unique ID of pet\n",
    "- __Outcome Type__ - State of pet at the time of recording the outcome (0 = not placed, 1 = placed). This is the field to predict.\n",
    "- __Sex upon Outcome__ - Sex of pet at outcome\n",
    "- __Name__ - Name of pet \n",
    "- __Found Location__ - Found location of pet before entered the center\n",
    "- __Intake Type__ - Circumstances bringing the pet to the center\n",
    "- __Intake Condition__ - Health condition of pet when entered the center\n",
    "- __Pet Type__ - Type of pet\n",
    "- __Sex upon Intake__ - Sex of pet when entered the center\n",
    "- __Breed__ - Breed of pet \n",
    "- __Color__ - Color of pet \n",
    "- __Age upon Intake Days__ - Age of pet when entered the center (days)\n",
    "- __Age upon Outcome Days__ - Age of pet at outcome (days)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "%pip install -q -r ../requirements.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. <a name=\"1\">Read the dataset</a>\n",
    "(<a href=\"#0\">Go to top</a>)\n",
    "\n",
    "Let's read the dataset into a dataframe, using Pandas."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The shape of the dataset is: (95485, 13)\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "  \n",
    "df = pd.read_csv('../data/review/review_dataset.csv')\n",
    "\n",
    "print('The shape of the dataset is:', df.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. <a name=\"2\">Exploratory Data Analysis</a>\n",
    "(<a href=\"#0\">Go to top</a>)\n",
    "\n",
    "We will look at number of rows, columns and some simple statistics of the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Pet ID</th>\n",
       "      <th>Outcome Type</th>\n",
       "      <th>Sex upon Outcome</th>\n",
       "      <th>Name</th>\n",
       "      <th>Found Location</th>\n",
       "      <th>Intake Type</th>\n",
       "      <th>Intake Condition</th>\n",
       "      <th>Pet Type</th>\n",
       "      <th>Sex upon Intake</th>\n",
       "      <th>Breed</th>\n",
       "      <th>Color</th>\n",
       "      <th>Age upon Intake Days</th>\n",
       "      <th>Age upon Outcome Days</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A794011</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Neutered Male</td>\n",
       "      <td>Chunk</td>\n",
       "      <td>Austin (TX)</td>\n",
       "      <td>Owner Surrender</td>\n",
       "      <td>Normal</td>\n",
       "      <td>Cat</td>\n",
       "      <td>Neutered Male</td>\n",
       "      <td>Domestic Shorthair Mix</td>\n",
       "      <td>Brown Tabby/White</td>\n",
       "      <td>730</td>\n",
       "      <td>730</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>A776359</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Neutered Male</td>\n",
       "      <td>Gizmo</td>\n",
       "      <td>7201 Levander Loop in Austin (TX)</td>\n",
       "      <td>Stray</td>\n",
       "      <td>Normal</td>\n",
       "      <td>Dog</td>\n",
       "      <td>Intact Male</td>\n",
       "      <td>Chihuahua Shorthair Mix</td>\n",
       "      <td>White/Brown</td>\n",
       "      <td>365</td>\n",
       "      <td>365</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>A674754</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Intact Male</td>\n",
       "      <td>NaN</td>\n",
       "      <td>12034 Research in Austin (TX)</td>\n",
       "      <td>Stray</td>\n",
       "      <td>Nursing</td>\n",
       "      <td>Cat</td>\n",
       "      <td>Intact Male</td>\n",
       "      <td>Domestic Shorthair Mix</td>\n",
       "      <td>Orange Tabby</td>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>A689724</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Neutered Male</td>\n",
       "      <td>*Donatello</td>\n",
       "      <td>2300 Waterway Bnd in Austin (TX)</td>\n",
       "      <td>Stray</td>\n",
       "      <td>Normal</td>\n",
       "      <td>Cat</td>\n",
       "      <td>Intact Male</td>\n",
       "      <td>Domestic Shorthair Mix</td>\n",
       "      <td>Black</td>\n",
       "      <td>60</td>\n",
       "      <td>60</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>A680969</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Neutered Male</td>\n",
       "      <td>*Zeus</td>\n",
       "      <td>4701 Staggerbrush Rd in Austin (TX)</td>\n",
       "      <td>Stray</td>\n",
       "      <td>Nursing</td>\n",
       "      <td>Cat</td>\n",
       "      <td>Intact Male</td>\n",
       "      <td>Domestic Shorthair Mix</td>\n",
       "      <td>White/Orange Tabby</td>\n",
       "      <td>7</td>\n",
       "      <td>60</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Pet ID  Outcome Type Sex upon Outcome        Name  \\\n",
       "0  A794011           1.0    Neutered Male       Chunk   \n",
       "1  A776359           1.0    Neutered Male       Gizmo   \n",
       "2  A674754           0.0      Intact Male         NaN   \n",
       "3  A689724           1.0    Neutered Male  *Donatello   \n",
       "4  A680969           1.0    Neutered Male       *Zeus   \n",
       "\n",
       "                        Found Location      Intake Type Intake Condition  \\\n",
       "0                          Austin (TX)  Owner Surrender           Normal   \n",
       "1    7201 Levander Loop in Austin (TX)            Stray           Normal   \n",
       "2        12034 Research in Austin (TX)            Stray          Nursing   \n",
       "3     2300 Waterway Bnd in Austin (TX)            Stray           Normal   \n",
       "4  4701 Staggerbrush Rd in Austin (TX)            Stray          Nursing   \n",
       "\n",
       "  Pet Type Sex upon Intake                    Breed               Color  \\\n",
       "0      Cat   Neutered Male   Domestic Shorthair Mix   Brown Tabby/White   \n",
       "1      Dog     Intact Male  Chihuahua Shorthair Mix         White/Brown   \n",
       "2      Cat     Intact Male   Domestic Shorthair Mix        Orange Tabby   \n",
       "3      Cat     Intact Male   Domestic Shorthair Mix               Black   \n",
       "4      Cat     Intact Male   Domestic Shorthair Mix  White/Orange Tabby   \n",
       "\n",
       "   Age upon Intake Days  Age upon Outcome Days  \n",
       "0                   730                    730  \n",
       "1                   365                    365  \n",
       "2                     6                      6  \n",
       "3                    60                     60  \n",
       "4                     7                     60  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Print the first five rows\n",
    "# NaN means missing data\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 95485 entries, 0 to 95484\n",
      "Data columns (total 13 columns):\n",
      " #   Column                 Non-Null Count  Dtype  \n",
      "---  ------                 --------------  -----  \n",
      " 0   Pet ID                 95485 non-null  object \n",
      " 1   Outcome Type           95485 non-null  float64\n",
      " 2   Sex upon Outcome       95484 non-null  object \n",
      " 3   Name                   59138 non-null  object \n",
      " 4   Found Location         95485 non-null  object \n",
      " 5   Intake Type            95485 non-null  object \n",
      " 6   Intake Condition       95485 non-null  object \n",
      " 7   Pet Type               95485 non-null  object \n",
      " 8   Sex upon Intake        95484 non-null  object \n",
      " 9   Breed                  95485 non-null  object \n",
      " 10  Color                  95485 non-null  object \n",
      " 11  Age upon Intake Days   95485 non-null  int64  \n",
      " 12  Age upon Outcome Days  95485 non-null  int64  \n",
      "dtypes: float64(1), int64(2), object(10)\n",
      "memory usage: 9.5+ MB\n"
     ]
    }
   ],
   "source": [
    "# Let's see the data types and non-null values for each column\n",
    "df.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Outcome Type</th>\n",
       "      <th>Age upon Intake Days</th>\n",
       "      <th>Age upon Outcome Days</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>95485.000000</td>\n",
       "      <td>95485.000000</td>\n",
       "      <td>95485.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.564005</td>\n",
       "      <td>703.436959</td>\n",
       "      <td>717.757313</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.495889</td>\n",
       "      <td>1052.252197</td>\n",
       "      <td>1055.023160</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>30.000000</td>\n",
       "      <td>60.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>365.000000</td>\n",
       "      <td>365.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>730.000000</td>\n",
       "      <td>730.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>9125.000000</td>\n",
       "      <td>9125.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       Outcome Type  Age upon Intake Days  Age upon Outcome Days\n",
       "count  95485.000000          95485.000000           95485.000000\n",
       "mean       0.564005            703.436959             717.757313\n",
       "std        0.495889           1052.252197            1055.023160\n",
       "min        0.000000              0.000000               0.000000\n",
       "25%        0.000000             30.000000              60.000000\n",
       "50%        1.000000            365.000000             365.000000\n",
       "75%        1.000000            730.000000             730.000000\n",
       "max        1.000000           9125.000000            9125.000000"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# This prints basic statistics for numerical columns\n",
    "df.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's separate model features and model target. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Index(['Pet ID', 'Outcome Type', 'Sex upon Outcome', 'Name', 'Found Location',\n",
      "       'Intake Type', 'Intake Condition', 'Pet Type', 'Sex upon Intake',\n",
      "       'Breed', 'Color', 'Age upon Intake Days', 'Age upon Outcome Days'],\n",
      "      dtype='object')\n"
     ]
    }
   ],
   "source": [
    "print(df.columns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model features:  Index(['Pet ID', 'Sex upon Outcome', 'Name', 'Found Location', 'Intake Type',\n",
      "       'Intake Condition', 'Pet Type', 'Sex upon Intake', 'Breed', 'Color',\n",
      "       'Age upon Intake Days', 'Age upon Outcome Days'],\n",
      "      dtype='object')\n",
      "Model target:  Outcome Type\n"
     ]
    }
   ],
   "source": [
    "model_features = df.columns.drop('Outcome Type')\n",
    "model_target = 'Outcome Type'\n",
    "\n",
    "print('Model features: ', model_features)\n",
    "print('Model target: ', model_target)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can explore the features set further, figuring out first what features are numerical or categorical. Beware that some integer-valued features could actually be categorical features, and some categorical features could be text features. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Numerical columns: Index(['Age upon Intake Days', 'Age upon Outcome Days'], dtype='object')\n",
      "\n",
      "Categorical columns: Index(['Pet ID', 'Sex upon Outcome', 'Name', 'Found Location', 'Intake Type',\n",
      "       'Intake Condition', 'Pet Type', 'Sex upon Intake', 'Breed', 'Color'],\n",
      "      dtype='object')\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "numerical_features_all = df[model_features].select_dtypes(include=np.number).columns\n",
    "print('Numerical columns:',numerical_features_all)\n",
    "\n",
    "print('')\n",
    "\n",
    "categorical_features_all = df[model_features].select_dtypes(include='object').columns\n",
    "print('Categorical columns:',categorical_features_all)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Target distribution\n",
    "\n",
    "Let's check our target distribution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjoAAAG5CAYAAACHhJ4rAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAsN0lEQVR4nO3df1iUdb7/8dcEQkAwIQTjnEitiJXQPUUdREstBfQCrdMf6qEz5eZiRUlscNV67DrZXgWl+eO0nDzkZpZa+Edr21mNoM5ZNo7iD1z25I/cfmjiyoCtOKixA9H9/WO/3mcHyAJbkQ/Px3XNde3c93tmPve0g8/rZmZwWJZlCQAAwECXDPQCAAAA/lYIHQAAYCxCBwAAGIvQAQAAxiJ0AACAsQgdAABgLEIHAAAYi9ABAADGCh7oBQykr7/+WseOHVNkZKQcDsdALwcAAHwHlmXp1KlTcrvduuSSc5+zGdKhc+zYMSUkJAz0MgAAQD80NjbqyiuvPOfMkA6dyMhISX95oqKiogZ4NQAA4Ltoa2tTQkKC/e/4uQzp0Dn766qoqChCBwCAQea7vO2ENyMDAABjEToAAMBYhA4AADAWoQMAAIxF6AAAAGMROgAAwFiEDgAAMBahAwAAjEXoAAAAYxE6AADAWIQOAAAwFqEDAACMRegAAABjEToAAMBYhA4AADBW8EAvAANj1E+3DPQScAEdfjZ7oJcAAAOCMzoAAMBYhA4AADAWoQMAAIxF6AAAAGMROgAAwFiEDgAAMBahAwAAjEXoAAAAYxE6AADAWIQOAAAwFqEDAACMRegAAABjEToAAMBYhA4AADAWoQMAAIxF6AAAAGMROgAAwFiEDgAAMBahAwAAjNWn0FmyZIkcDkfAxeVy2fsty9KSJUvkdrsVFhamKVOmaN++fQH34ff7tXDhQsXGxioiIkKzZs3S0aNHA2ZaW1vl8XjkdDrldDrl8Xh08uTJgJkjR45o5syZioiIUGxsrAoKCtTR0dHHwwcAACbr8xmd66+/Xk1NTfblww8/tPctXbpUK1asUFlZmXbt2iWXy6WMjAydOnXKniksLNTmzZtVUVGh2tpanT59Wjk5Oerq6rJncnNz1dDQoMrKSlVWVqqhoUEej8fe39XVpezsbJ05c0a1tbWqqKjQm2++qaKiov4+DwAAwEDBfb5BcHDAWZyzLMvSqlWrtHjxYt11112SpFdffVXx8fF6/fXXdf/998vn8+nll1/W+vXrNW3aNEnShg0blJCQoPfee09ZWVk6cOCAKisrVVdXp7S0NEnSmjVrlJ6eroMHDyopKUlVVVXav3+/Ghsb5Xa7JUnLly/XvHnz9MwzzygqKqrfTwgAADBHn8/ofPzxx3K73Ro9erTmzp2rzz77TJJ06NAheb1eZWZm2rOhoaGaPHmytm3bJkmqr69XZ2dnwIzb7VZKSoo9s337djmdTjtyJGn8+PFyOp0BMykpKXbkSFJWVpb8fr/q6+v7ekgAAMBQfTqjk5aWptdee03XXXedmpub9fTTT2vChAnat2+fvF6vJCk+Pj7gNvHx8fr8888lSV6vVyEhIYqOju4xc/b2Xq9XcXFxPR47Li4uYKb740RHRyskJMSe6Y3f75ff77evt7W1fddDBwAAg1CfQmfGjBn2/x47dqzS09N1zTXX6NVXX9X48eMlSQ6HI+A2lmX12NZd95ne5vsz011paameeuqpc64FAACY47w+Xh4REaGxY8fq448/tt+30/2MSktLi332xeVyqaOjQ62treecaW5u7vFYx48fD5jp/jitra3q7Ozscabnry1atEg+n8++NDY29vGIAQDAYHJeoeP3+3XgwAGNGDFCo0ePlsvlUnV1tb2/o6NDNTU1mjBhgiQpNTVVw4YNC5hpamrS3r177Zn09HT5fD7t3LnTntmxY4d8Pl/AzN69e9XU1GTPVFVVKTQ0VKmpqd+43tDQUEVFRQVcAACAufr0q6vi4mLNnDlTV111lVpaWvT000+rra1N9957rxwOhwoLC1VSUqLExEQlJiaqpKRE4eHhys3NlSQ5nU7Nnz9fRUVFiomJ0fDhw1VcXKyxY8fan8IaM2aMpk+frry8PJWXl0uSFixYoJycHCUlJUmSMjMzlZycLI/Ho2XLlunEiRMqLi5WXl4e8QIAAGx9Cp2jR4/qn/7pn/TFF1/oiiuu0Pjx41VXV6eRI0dKkh577DG1t7crPz9fra2tSktLU1VVlSIjI+37WLlypYKDgzV79my1t7dr6tSpWrdunYKCguyZjRs3qqCgwP501qxZs1RWVmbvDwoK0pYtW5Sfn6+JEycqLCxMubm5ev7558/ryQAAAGZxWJZlDfQiBkpbW5ucTqd8Pt+QOxM06qdbBnoJuIAOP5s90EsAgO9NX/795m9dAQAAYxE6AADAWIQOAAAwFqEDAACMRegAAABjEToAAMBYhA4AADAWoQMAAIxF6AAAAGMROgAAwFiEDgAAMBahAwAAjEXoAAAAYxE6AADAWIQOAAAwFqEDAACMRegAAABjEToAAMBYhA4AADAWoQMAAIxF6AAAAGMROgAAwFiEDgAAMBahAwAAjEXoAAAAYxE6AADAWIQOAAAwFqEDAACMRegAAABjEToAAMBYhA4AADAWoQMAAIxF6AAAAGMROgAAwFiEDgAAMBahAwAAjEXoAAAAYxE6AADAWIQOAAAwFqEDAACMRegAAABjEToAAMBYhA4AADAWoQMAAIxF6AAAAGMROgAAwFiEDgAAMBahAwAAjBU80AsAAHy/Rv10y0AvARfQ4WezB3oJFzXO6AAAAGMROgAAwFiEDgAAMBahAwAAjEXoAAAAYxE6AADAWOcVOqWlpXI4HCosLLS3WZalJUuWyO12KywsTFOmTNG+ffsCbuf3+7Vw4ULFxsYqIiJCs2bN0tGjRwNmWltb5fF45HQ65XQ65fF4dPLkyYCZI0eOaObMmYqIiFBsbKwKCgrU0dFxPocEAAAM0u/Q2bVrl1566SWNGzcuYPvSpUu1YsUKlZWVadeuXXK5XMrIyNCpU6fsmcLCQm3evFkVFRWqra3V6dOnlZOTo66uLnsmNzdXDQ0NqqysVGVlpRoaGuTxeOz9XV1dys7O1pkzZ1RbW6uKigq9+eabKioq6u8hAQAAw/QrdE6fPq27775ba9asUXR0tL3dsiytWrVKixcv1l133aWUlBS9+uqr+vLLL/X6669Lknw+n15++WUtX75c06ZN0w033KANGzboww8/1HvvvSdJOnDggCorK/WLX/xC6enpSk9P15o1a/TrX/9aBw8elCRVVVVp//792rBhg2644QZNmzZNy5cv15o1a9TW1na+zwsAADBAv0LnoYceUnZ2tqZNmxaw/dChQ/J6vcrMzLS3hYaGavLkydq2bZskqb6+Xp2dnQEzbrdbKSkp9sz27dvldDqVlpZmz4wfP15OpzNgJiUlRW63257JysqS3+9XfX19r+v2+/1qa2sLuAAAAHP1+U9AVFRUaM+ePdq1a1ePfV6vV5IUHx8fsD0+Pl6ff/65PRMSEhJwJujszNnbe71excXF9bj/uLi4gJnujxMdHa2QkBB7prvS0lI99dRT3+UwAQCAAfp0RqexsVGPPPKINmzYoEsvvfQb5xwOR8B1y7J6bOuu+0xv8/2Z+WuLFi2Sz+ezL42NjedcEwAAGNz6FDr19fVqaWlRamqqgoODFRwcrJqaGr3wwgsKDg62z7B0P6PS0tJi73O5XOro6FBra+s5Z5qbm3s8/vHjxwNmuj9Oa2urOjs7e5zpOSs0NFRRUVEBFwAAYK4+hc7UqVP14YcfqqGhwb7cdNNNuvvuu9XQ0KCrr75aLpdL1dXV9m06OjpUU1OjCRMmSJJSU1M1bNiwgJmmpibt3bvXnklPT5fP59POnTvtmR07dsjn8wXM7N27V01NTfZMVVWVQkNDlZqa2o+nAgAAmKZP79GJjIxUSkpKwLaIiAjFxMTY2wsLC1VSUqLExEQlJiaqpKRE4eHhys3NlSQ5nU7Nnz9fRUVFiomJ0fDhw1VcXKyxY8fab24eM2aMpk+frry8PJWXl0uSFixYoJycHCUlJUmSMjMzlZycLI/Ho2XLlunEiRMqLi5WXl4eZ2oAAICkfrwZ+ds89thjam9vV35+vlpbW5WWlqaqqipFRkbaMytXrlRwcLBmz56t9vZ2TZ06VevWrVNQUJA9s3HjRhUUFNifzpo1a5bKysrs/UFBQdqyZYvy8/M1ceJEhYWFKTc3V88///z3fUgAAGCQcliWZQ30IgZKW1ubnE6nfD7fkDsLNOqnWwZ6CbiADj+bPdBLwAXE63toGYqv7778+83fugIAAMYidAAAgLEIHQAAYCxCBwAAGIvQAQAAxiJ0AACAsQgdAABgLEIHAAAYi9ABAADGInQAAICxCB0AAGAsQgcAABiL0AEAAMYidAAAgLEIHQAAYCxCBwAAGIvQAQAAxiJ0AACAsQgdAABgLEIHAAAYi9ABAADGInQAAICxCB0AAGAsQgcAABiL0AEAAMYidAAAgLEIHQAAYCxCBwAAGIvQAQAAxiJ0AACAsQgdAABgLEIHAAAYi9ABAADGInQAAICxCB0AAGAsQgcAABiL0AEAAMYidAAAgLEIHQAAYCxCBwAAGIvQAQAAxiJ0AACAsQgdAABgLEIHAAAYi9ABAADGInQAAICxCB0AAGAsQgcAABiL0AEAAMYidAAAgLEIHQAAYCxCBwAAGIvQAQAAxiJ0AACAsfoUOqtXr9a4ceMUFRWlqKgopaen65133rH3W5alJUuWyO12KywsTFOmTNG+ffsC7sPv92vhwoWKjY1VRESEZs2apaNHjwbMtLa2yuPxyOl0yul0yuPx6OTJkwEzR44c0cyZMxUREaHY2FgVFBSoo6Ojj4cPAABM1qfQufLKK/Xss89q9+7d2r17t26//XbdcccddswsXbpUK1asUFlZmXbt2iWXy6WMjAydOnXKvo/CwkJt3rxZFRUVqq2t1enTp5WTk6Ouri57Jjc3Vw0NDaqsrFRlZaUaGhrk8Xjs/V1dXcrOztaZM2dUW1uriooKvfnmmyoqKjrf5wMAABjEYVmWdT53MHz4cC1btkz33Xef3G63CgsL9fjjj0v6y9mb+Ph4Pffcc7r//vvl8/l0xRVXaP369ZozZ44k6dixY0pISNDWrVuVlZWlAwcOKDk5WXV1dUpLS5Mk1dXVKT09XR999JGSkpL0zjvvKCcnR42NjXK73ZKkiooKzZs3Ty0tLYqKivpOa29ra5PT6ZTP5/vOtzHFqJ9uGegl4AI6/Gz2QC8BFxCv76FlKL6++/Lvd7/fo9PV1aWKigqdOXNG6enpOnTokLxerzIzM+2Z0NBQTZ48Wdu2bZMk1dfXq7OzM2DG7XYrJSXFntm+fbucTqcdOZI0fvx4OZ3OgJmUlBQ7ciQpKytLfr9f9fX1/T0kAABgmOC+3uDDDz9Uenq6/vznP+uyyy7T5s2blZycbEdIfHx8wHx8fLw+//xzSZLX61VISIiio6N7zHi9XnsmLi6ux+PGxcUFzHR/nOjoaIWEhNgzvfH7/fL7/fb1tra273rYAABgEOrzGZ2kpCQ1NDSorq5ODz74oO69917t37/f3u9wOALmLcvqsa277jO9zfdnprvS0lL7Dc5Op1MJCQnnXBcAABjc+hw6ISEhuvbaa3XTTTeptLRUP/zhD/Vv//ZvcrlcktTjjEpLS4t99sXlcqmjo0Otra3nnGlubu7xuMePHw+Y6f44ra2t6uzs7HGm568tWrRIPp/PvjQ2Nvbx6AEAwGBy3t+jY1mW/H6/Ro8eLZfLperqantfR0eHampqNGHCBElSamqqhg0bFjDT1NSkvXv32jPp6eny+XzauXOnPbNjxw75fL6Amb1796qpqcmeqaqqUmhoqFJTU79xraGhofZH489eAACAufr0Hp1/+Zd/0YwZM5SQkKBTp06poqJCv/nNb1RZWSmHw6HCwkKVlJQoMTFRiYmJKikpUXh4uHJzcyVJTqdT8+fPV1FRkWJiYjR8+HAVFxdr7NixmjZtmiRpzJgxmj59uvLy8lReXi5JWrBggXJycpSUlCRJyszMVHJysjwej5YtW6YTJ06ouLhYeXl5xAsAALD1KXSam5vl8XjU1NQkp9OpcePGqbKyUhkZGZKkxx57TO3t7crPz1dra6vS0tJUVVWlyMhI+z5Wrlyp4OBgzZ49W+3t7Zo6darWrVunoKAge2bjxo0qKCiwP501a9YslZWV2fuDgoK0ZcsW5efna+LEiQoLC1Nubq6ef/7583oyAACAWc77e3QGM75HB0PFUPyejaGM1/fQMhRf3xfke3QAAAAudoQOAAAwFqEDAACMRegAAABjEToAAMBYhA4AADAWoQMAAIxF6AAAAGMROgAAwFiEDgAAMBahAwAAjEXoAAAAYxE6AADAWIQOAAAwFqEDAACMRegAAABjEToAAMBYhA4AADAWoQMAAIxF6AAAAGMROgAAwFiEDgAAMBahAwAAjEXoAAAAYxE6AADAWIQOAAAwFqEDAACMRegAAABjEToAAMBYhA4AADAWoQMAAIxF6AAAAGMROgAAwFiEDgAAMBahAwAAjEXoAAAAYxE6AADAWIQOAAAwFqEDAACMRegAAABjEToAAMBYhA4AADAWoQMAAIxF6AAAAGMROgAAwFiEDgAAMBahAwAAjEXoAAAAYxE6AADAWIQOAAAwFqEDAACMRegAAABjEToAAMBYhA4AADBWn0KntLRUN998syIjIxUXF6c777xTBw8eDJixLEtLliyR2+1WWFiYpkyZon379gXM+P1+LVy4ULGxsYqIiNCsWbN09OjRgJnW1lZ5PB45nU45nU55PB6dPHkyYObIkSOaOXOmIiIiFBsbq4KCAnV0dPTlkAAAgMH6FDo1NTV66KGHVFdXp+rqan311VfKzMzUmTNn7JmlS5dqxYoVKisr065du+RyuZSRkaFTp07ZM4WFhdq8ebMqKipUW1ur06dPKycnR11dXfZMbm6uGhoaVFlZqcrKSjU0NMjj8dj7u7q6lJ2drTNnzqi2tlYVFRV68803VVRUdD7PBwAAMIjDsiyrvzc+fvy44uLiVFNTo0mTJsmyLLndbhUWFurxxx+X9JezN/Hx8Xruued0//33y+fz6YorrtD69es1Z84cSdKxY8eUkJCgrVu3KisrSwcOHFBycrLq6uqUlpYmSaqrq1N6ero++ugjJSUl6Z133lFOTo4aGxvldrslSRUVFZo3b55aWloUFRX1retva2uT0+mUz+f7TvMmGfXTLQO9BFxAh5/NHugl4ALi9T20DMXXd1/+/T6v9+j4fD5J0vDhwyVJhw4dktfrVWZmpj0TGhqqyZMna9u2bZKk+vp6dXZ2Bsy43W6lpKTYM9u3b5fT6bQjR5LGjx8vp9MZMJOSkmJHjiRlZWXJ7/ervr7+fA4LAAAYIri/N7QsS48++qhuueUWpaSkSJK8Xq8kKT4+PmA2Pj5en3/+uT0TEhKi6OjoHjNnb+/1ehUXF9fjMePi4gJmuj9OdHS0QkJC7Jnu/H6//H6/fb2tre07Hy8AABh8+n1G5+GHH9b//u//6o033uixz+FwBFy3LKvHtu66z/Q235+Zv1ZaWmq/udnpdCohIeGcawIAAINbv0Jn4cKFevvtt/Xf//3fuvLKK+3tLpdLknqcUWlpabHPvrhcLnV0dKi1tfWcM83NzT0e9/jx4wEz3R+ntbVVnZ2dPc70nLVo0SL5fD770tjY2JfDBgAAg0yfQseyLD388MP65S9/qf/6r//S6NGjA/aPHj1aLpdL1dXV9raOjg7V1NRowoQJkqTU1FQNGzYsYKapqUl79+61Z9LT0+Xz+bRz5057ZseOHfL5fAEze/fuVVNTkz1TVVWl0NBQpaam9rr+0NBQRUVFBVwAAIC5+vQenYceekivv/66fvWrXykyMtI+o+J0OhUWFiaHw6HCwkKVlJQoMTFRiYmJKikpUXh4uHJzc+3Z+fPnq6ioSDExMRo+fLiKi4s1duxYTZs2TZI0ZswYTZ8+XXl5eSovL5ckLViwQDk5OUpKSpIkZWZmKjk5WR6PR8uWLdOJEydUXFysvLw8AgYAAEjqY+isXr1akjRlypSA7a+88ormzZsnSXrsscfU3t6u/Px8tba2Ki0tTVVVVYqMjLTnV65cqeDgYM2ePVvt7e2aOnWq1q1bp6CgIHtm48aNKigosD+dNWvWLJWVldn7g4KCtGXLFuXn52vixIkKCwtTbm6unn/++T49AQAAwFzn9T06gx3fo4OhYih+z8ZQxut7aBmKr+8L9j06AAAAFzNCBwAAGIvQAQAAxiJ0AACAsQgdAABgLEIHAAAYi9ABAADGInQAAICxCB0AAGAsQgcAABiL0AEAAMYidAAAgLEIHQAAYCxCBwAAGIvQAQAAxiJ0AACAsQgdAABgLEIHAAAYi9ABAADGInQAAICxCB0AAGAsQgcAABiL0AEAAMYidAAAgLEIHQAAYCxCBwAAGIvQAQAAxiJ0AACAsQgdAABgLEIHAAAYi9ABAADGInQAAICxCB0AAGAsQgcAABiL0AEAAMYidAAAgLEIHQAAYCxCBwAAGIvQAQAAxiJ0AACAsQgdAABgLEIHAAAYi9ABAADGInQAAICxCB0AAGAsQgcAABiL0AEAAMYidAAAgLEIHQAAYCxCBwAAGIvQAQAAxiJ0AACAsQgdAABgLEIHAAAYi9ABAADG6nPo/Pa3v9XMmTPldrvlcDj01ltvBey3LEtLliyR2+1WWFiYpkyZon379gXM+P1+LVy4ULGxsYqIiNCsWbN09OjRgJnW1lZ5PB45nU45nU55PB6dPHkyYObIkSOaOXOmIiIiFBsbq4KCAnV0dPT1kAAAgKH6HDpnzpzRD3/4Q5WVlfW6f+nSpVqxYoXKysq0a9cuuVwuZWRk6NSpU/ZMYWGhNm/erIqKCtXW1ur06dPKyclRV1eXPZObm6uGhgZVVlaqsrJSDQ0N8ng89v6uri5lZ2frzJkzqq2tVUVFhd58800VFRX19ZAAAIChgvt6gxkzZmjGjBm97rMsS6tWrdLixYt11113SZJeffVVxcfH6/XXX9f9998vn8+nl19+WevXr9e0adMkSRs2bFBCQoLee+89ZWVl6cCBA6qsrFRdXZ3S0tIkSWvWrFF6eroOHjyopKQkVVVVaf/+/WpsbJTb7ZYkLV++XPPmzdMzzzyjqKiofj0hAADAHN/re3QOHTokr9erzMxMe1toaKgmT56sbdu2SZLq6+vV2dkZMON2u5WSkmLPbN++XU6n044cSRo/frycTmfATEpKih05kpSVlSW/36/6+vpe1+f3+9XW1hZwAQAA5vpeQ8fr9UqS4uPjA7bHx8fb+7xer0JCQhQdHX3Ombi4uB73HxcXFzDT/XGio6MVEhJiz3RXWlpqv+fH6XQqISGhH0cJAAAGi7/Jp64cDkfAdcuyemzrrvtMb/P9mflrixYtks/nsy+NjY3nXBMAABjcvtfQcblcktTjjEpLS4t99sXlcqmjo0Otra3nnGlubu5x/8ePHw+Y6f44ra2t6uzs7HGm56zQ0FBFRUUFXAAAgLm+19AZPXq0XC6Xqqur7W0dHR2qqanRhAkTJEmpqakaNmxYwExTU5P27t1rz6Snp8vn82nnzp32zI4dO+Tz+QJm9u7dq6amJnumqqpKoaGhSk1N/T4PCwAADFJ9/tTV6dOn9cknn9jXDx06pIaGBg0fPlxXXXWVCgsLVVJSosTERCUmJqqkpETh4eHKzc2VJDmdTs2fP19FRUWKiYnR8OHDVVxcrLFjx9qfwhozZoymT5+uvLw8lZeXS5IWLFignJwcJSUlSZIyMzOVnJwsj8ejZcuW6cSJEyouLlZeXh5nagAAgKR+hM7u3bt122232dcfffRRSdK9996rdevW6bHHHlN7e7vy8/PV2tqqtLQ0VVVVKTIy0r7NypUrFRwcrNmzZ6u9vV1Tp07VunXrFBQUZM9s3LhRBQUF9qezZs2aFfDdPUFBQdqyZYvy8/M1ceJEhYWFKTc3V88//3zfnwUAAGAkh2VZ1kAvYqC0tbXJ6XTK5/MNubNAo366ZaCXgAvo8LPZA70EXEC8voeWofj67su/3/ytKwAAYCxCBwAAGIvQAQAAxiJ0AACAsQgdAABgLEIHAAAYi9ABAADGInQAAICxCB0AAGAsQgcAABiL0AEAAMYidAAAgLEIHQAAYCxCBwAAGIvQAQAAxiJ0AACAsQgdAABgLEIHAAAYi9ABAADGInQAAICxCB0AAGAsQgcAABiL0AEAAMYidAAAgLEIHQAAYCxCBwAAGIvQAQAAxiJ0AACAsQgdAABgLEIHAAAYi9ABAADGInQAAICxCB0AAGAsQgcAABiL0AEAAMYidAAAgLEIHQAAYCxCBwAAGIvQAQAAxiJ0AACAsQgdAABgLEIHAAAYi9ABAADGInQAAICxCB0AAGAsQgcAABiL0AEAAMYidAAAgLEIHQAAYCxCBwAAGIvQAQAAxiJ0AACAsQgdAABgLEIHAAAYa9CHzosvvqjRo0fr0ksvVWpqqj744IOBXhIAALhIDOrQ2bRpkwoLC7V48WL97ne/06233qoZM2boyJEjA700AABwERjUobNixQrNnz9fP/7xjzVmzBitWrVKCQkJWr169UAvDQAAXAQGbeh0dHSovr5emZmZAdszMzO1bdu2AVoVAAC4mAQP9AL664svvlBXV5fi4+MDtsfHx8vr9fZ6G7/fL7/fb1/3+XySpLa2tr/dQi9SX/u/HOgl4AIaiv8fH8p4fQ8tQ/H1ffaYLcv61tlBGzpnORyOgOuWZfXYdlZpaameeuqpHtsTEhL+JmsDLhbOVQO9AgB/K0P59X3q1Ck5nc5zzgza0ImNjVVQUFCPszctLS09zvKctWjRIj366KP29a+//lonTpxQTEzMN8YRzNHW1qaEhAQ1NjYqKipqoJcD4HvE63tosSxLp06dktvt/tbZQRs6ISEhSk1NVXV1tf7xH//R3l5dXa077rij19uEhoYqNDQ0YNvll1/+t1wmLkJRUVH8IAQMxet76Pi2MzlnDdrQkaRHH31UHo9HN910k9LT0/XSSy/pyJEjeuCBBwZ6aQAA4CIwqENnzpw5+tOf/qSf/exnampqUkpKirZu3aqRI0cO9NIAAMBFYFCHjiTl5+crPz9/oJeBQSA0NFRPPvlkj19fAhj8eH3jmzis7/LZLAAAgEFo0H5hIAAAwLchdAAAgLEIHQAAYCxCBwAAGGvQf+oKADB0dXV16YsvvpDD4VBMTIyCgoIGekm4yHBGB0NCV1eXmpub1dLSoq6uroFeDoDztHnzZk2cOFHh4eFyu90aMWKEwsPDNXHiRL311lsDvTxcRAgdGI0fhoB5ysvLNXfuXI0bN06bNm1SbW2tPvjgA23atEnjxo3T3LlztWbNmoFeJi4SfI8OjFVeXq6CggLdd999ysrKUnx8vCzLUktLi95991298sor+vnPf668vLyBXiqAPrj22mu1aNEizZ8/v9f9a9eu1TPPPKNPP/30Aq8MFyNCB8bihyFgprCwMDU0NCgpKanX/R999JFuuOEGtbe3X+CV4WLEr65grD/+8Y+65ZZbvnH/hAkTdOzYsQu4IgDfh+uvv14vvfTSN+5fs2aNrr/++gu4IlzM+NQVjHX2h+Hy5ct73c8PQ2BwWr58ubKzs1VZWanMzEzFx8fL4XDI6/Wqurpan3/+ubZu3TrQy8RFgl9dwVg1NTXKzs7WyJEjz/nD8NZbbx3opQLoo8OHD2v16tWqq6uT1+uVJLlcLqWnp+uBBx7QqFGjBnaBuGgQOjAaPwwBYGgjdAAAgLF4MzIAwCj33nuvbr/99oFeBi4ShA6GLH4YAmZyu90aOXLkQC8DFwk+dYUhy+1265JLaH3ANKWlpQO9BFxEeI8OAGDQOXr0qFavXq1t27bJ6/XK4XAoPj5eEyZM0IMPPqgrr7xyoJeIiwShgyGrsbFRTz75pNauXTvQSwHQB7W1tZoxY4YSEhLsr444++ddqqur1djYqHfeeUcTJ04c6KXiIkDoYMj6/e9/rxtvvJG/Zg4MMjfffLNuueUWrVy5stf9P/nJT1RbW6tdu3Zd4JXhYkTowFhvv/32Ofd/9tlnKioqInSAQYa/dYW+4M3IMNadd94ph8Ohc7W8w+G4gCsC8H0YMWKEtm3b9o2hs337do0YMeICrwoXK0IHxhoxYoT+/d//XXfeeWev+xsaGpSamnphFwXgvBUXF+uBBx5QfX29MjIyevx5l1/84hdatWrVQC8TFwlCB8ZKTU3Vnj17vjF0vu1sD4CLU35+vmJiYrRy5UqVl5fbv34OCgpSamqqXnvtNc2ePXuAV4mLBe/RgbE++OADnTlzRtOnT+91/5kzZ7R7925Nnjz5Aq8MwPels7NTX3zxhSQpNjZWw4YNG+AV4WJD6AAAAGPxtbAAAMBYhA4AADAWoQMAAIxF6AAAAGMROgAAwFiEDoBv1NjYqPnz58vtdiskJEQjR47UI488oj/96U99up/Dhw/L4XCooaHhb7PQv6F169bJ4XCc8/Kb3/xmoJcJ4BsQOgB69dlnn+mmm27SH/7wB73xxhv65JNP9B//8R96//33lZ6erhMnTgz0Ei+IOXPmqKmpyb6kp6crLy8vYNuECRMGepkAvgGhA6BXDz30kEJCQlRVVaXJkyfrqquu0owZM/Tee+/pj3/8oxYvXmzPOhwOvfXWWwG3v/zyy7Vu3TpJ0ujRoyVJN9xwgxwOh6ZMmWLPrV27Vtdff71CQ0M1YsQIPfzww/a+I0eO6I477tBll12mqKgozZ49W83Nzfb+JUuW6O///u+1du1aXXXVVbrsssv04IMPqqurS0uXLpXL5VJcXJyeeeaZgLX5fD4tWLBAcXFxioqK0u23367f//73vT4PYWFhcrlc9iUkJETh4eFyuVz6wx/+oISEhB7RV1RUpEmTJkn6yxmhyy+/XG+99Zauu+46XXrppcrIyFBjY2PAbf7zP/9TqampuvTSS3X11Vfrqaee0ldffXWO/0IAvgtCB0APJ06c0Lvvvqv8/HyFhYUF7HO5XLr77ru1adOm7/wnNHbu3ClJeu+999TU1KRf/vKXkqTVq1froYce0oIFC/Thhx/q7bff1rXXXitJsixLd955p06cOKGamhpVV1fr008/1Zw5cwLu+9NPP9U777yjyspKvfHGG1q7dq2ys7N19OhR1dTU6LnnntMTTzyhuro6+36zs7Pl9Xq1detW1dfX68Ybb9TUqVP7fJZq0qRJuvrqq7V+/Xp721dffaUNGzboRz/6kb3tyy+/1DPPPKNXX31V//M//6O2tjbNnTvX3v/uu+/qn//5n1VQUKD9+/ervLxc69at6xFoAPrBAoBu6urqLEnW5s2be92/YsUKS5LV3NxsWZbV66zT6bReeeUVy7Is69ChQ5Yk63e/+13AjNvtthYvXtzrY1RVVVlBQUHWkSNH7G379u2zJFk7d+60LMuynnzySSs8PNxqa2uzZ7KysqxRo0ZZXV1d9rakpCSrtLTUsizLev/9962oqCjrz3/+c8DjXXPNNVZ5eXnvT8hfmTx5svXII4/Y15977jlrzJgx9vW33nrLuuyyy6zTp09blmVZr7zyiiXJqqurs2cOHDhgSbJ27NhhWZZl3XrrrVZJSUnA46xfv94aMWLEt64HwLlxRgdAn1n//0yOw+Ho9320tLTo2LFjmjp1aq/7Dxw4oISEBCUkJNjbkpOTdfnll+vAgQP2tlGjRikyMtK+Hh8fr+TkZF1yySUB21paWiRJ9fX1On36tGJiYnTZZZfZl0OHDunTTz/t83HMmzdPn3zyiX3GaO3atZo9e7YiIiLsmeDgYN1000329R/84AcBx1FfX6+f/exnAes5+z6gL7/8ss9rAvB/+OvlAHq49tpr5XA4tH///l7/+vtHH32k6OhoxcbGSur9L8F3dnae8zG6/0qsO8uyeg2p7tu7/xFHh8PR67avv/5akvT1119rxIgRvX5S6vLLLz/nmnoTFxenmTNn6pVXXtHVV1+trVu39nrfvR3L2W1ff/21nnrqKd111109Zi699NI+rwnA/yF0APQQExOjjIwMvfjii/rJT34SECVer1cbN27UPffcY/9DfcUVV6ipqcme+fjjjwPORISEhEiSurq67G2RkZEaNWqU3n//fd1222091pCcnKwjR46osbHRPquzf/9++Xw+jRkzpt/HduONN8rr9So4OFijRo3q9/38tR//+MeaO3eurrzySl1zzTWaOHFiwP6vvvpKu3fv1j/8wz9Ikg4ePKiTJ0/qBz/4gb2mgwcP2u9PAvD94VdXAHpVVlYmv9+vrKws/fa3v1VjY6MqKyuVkZGhv/u7vwt4o+ztt9+usrIy7dmzR7t379YDDzwQcFYlLi5OYWFhqqysVHNzs3w+n6S/fGpq+fLleuGFF/Txxx9rz549+vnPfy5JmjZtmsaNG6e7775be/bs0c6dO3XPPfdo8uTJAb8G6qtp06YpPT1dd955p959910dPnxY27Zt0xNPPKHdu3f36z6zsrLkdDr19NNPB7wJ+axhw4Zp4cKF2rFjh/bs2aMf/ehHGj9+vB0+//qv/6rXXntNS5Ys0b59+3TgwAFt2rRJTzzxRL+PE8BfEDoAepWYmKjdu3frmmuu0Zw5c3TNNddowYIFuu2227R9+3YNHz7cnl2+fLkSEhI0adIk5ebmqri4WOHh4fb+4OBgvfDCCyovL5fb7dYdd9whSbr33nu1atUqvfjii7r++uuVk5Ojjz/+WNL/fWQ9OjpakyZN0rRp03T11Vdr06ZN53VcDodDW7du1aRJk3Tffffpuuuu09y5c3X48GHFx8f36z4vueQSzZs3T11dXbrnnnt67A8PD9fjjz+u3NxcpaenKywsTBUVFfb+rKws/frXv1Z1dbVuvvlmjR8/XitWrNDIkSP7fZwA/sJhdf/FOgCgz/Ly8tTc3Ky33347YPu6detUWFiokydPDszCgCGO9+gAwHnw+XzatWuXNm7cqF/96lcDvRwA3RA6AHAe7rjjDu3cuVP333+/MjIyBno5ALrhV1cAAMBYvBkZAAAYi9ABAADGInQAAICxCB0AAGAsQgcAABiL0AEAAMYidAAAgLEIHQAAYCxCBwAAGOv/ASeiyMjPE4l5AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "df[model_target].value_counts().plot.bar()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the target plots we can identify whether or not we are dealing with imbalanced datasets - this means one result type is dominating the other one(s). \n",
    "\n",
    "Handling class imbalance is highly recommended, as the model performance can be greatly impacted. In particular the model may not work well for the infrequent classes, as there are not enough samples to learn patterns from, and so it would be hard for the classifier to identify and match those patterns. \n",
    "\n",
    "We might want to downsample the dominant class or upsample the rare the class, to help with learning its patterns. However, we should only fix the imbalance in training set, without changing the validation and test sets, as these should follow the original distribution. We will perform this task after train/test split. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. <a name=\"3\">Select features to build the model</a>\n",
    "(<a href=\"#0\">Go to top</a>)\n",
    "\n",
    "This time we build a model using all features. That is, we build a classifier including __numerical, categorical__ and __text__ features. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Grab model features/inputs and target/output\n",
    "\n",
    "# can also grab less numerical features, as some numerical data might not be very useful\n",
    "numerical_features = ['Age upon Intake Days', 'Age upon Outcome Days']\n",
    "\n",
    "# dropping the IDs features, RescuerID and PetID here \n",
    "categorical_features = ['Sex upon Outcome', 'Intake Type',\n",
    "       'Intake Condition', 'Pet Type', 'Sex upon Intake']\n",
    "\n",
    "# from EDA, select the text features\n",
    "text_features = ['Name', 'Found Location', 'Breed', 'Color']\n",
    "    \n",
    "model_features = numerical_features + categorical_features + text_features\n",
    "model_target = 'Outcome Type'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Cleaning numerical features "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Age upon Intake Days\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAk0AAAGdCAYAAAAPLEfqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAAA7p0lEQVR4nO3df1yV9cH/8TeBnJDBFYpwOouK7jGSsK1wQ7RNmwo2kFV7pEWddDm0USIL5o+1+zvrLvBX2hbLzPXQUou2mbvbTAb9mBtT1DBKzKwtE0wQy+NBjQ6E1/eP5nV3wOySoQfs9Xw8zh/nut7nuj7XuR7Gu8+5znWCTNM0BQAAgFM6L9ADAAAA6AsoTQAAADZQmgAAAGygNAEAANhAaQIAALCB0gQAAGADpQkAAMAGShMAAIANIYEewLnk+PHj2r9/vyIiIhQUFBTo4QAAABtM09SRI0fkcrl03nmfP59EaepB+/fvV1xcXKCHAQAAuqGhoUEXXXTR566nNPWgiIgISZ++6ZGRkQEeDQAAsKOlpUVxcXHW3/HPQ2nqQSc+kouMjKQ0AQDQx3zRpTVcCA4AAGADpQkAAMAGShMAAIANlCYAAAAbKE0AAAA2UJoAAABsoDQBAADYQGkCAACwgdIEAABgA6UJAADABkoTAACADZQmAAAAGyhNAAAANlCaAAAAbAgJ9ABgz6Wz1wd6CF8K783LDPQQAAC9FDNNAAAANlCaAAAAbKA0AQAA2EBpAgAAsIHSBAAAYAOlCQAAwAZKEwAAgA0BLU2ffPKJfvGLXyg+Pl5hYWG67LLLdP/99+v48eNWxjRNzZ07Vy6XS2FhYRo1apR27tzptx2fz6fp06crOjpa4eHhys7O1r59+/wyHo9HbrdbhmHIMAy53W4dPnzYL1NfX6/x48crPDxc0dHRys/PV1tb2xk7fgAA0HcEtDTNnz9fjz32mEpLS7Vr1y4tWLBACxcu1COPPGJlFixYoMWLF6u0tFTbtm2T0+nU2LFjdeTIEStTUFCgdevWqaysTFVVVTp69KiysrLU0dFhZXJyclRbW6vy8nKVl5ertrZWbrfbWt/R0aHMzEwdO3ZMVVVVKisr09q1a1VYWHh23gwAANCrBZmmaQZq51lZWYqNjdUTTzxhLfvhD3+o/v37a9WqVTJNUy6XSwUFBZo1a5akT2eVYmNjNX/+fE2bNk1er1eDBg3SqlWrNHHiREnS/v37FRcXpxdeeEEZGRnatWuXkpKSVF1drdTUVElSdXW10tLS9NZbbykxMVEbNmxQVlaWGhoa5HK5JEllZWWaPHmympubFRkZ+YXH09LSIsMw5PV6beVPB3cEPzu4IzgAfPnY/fsd0Jmma665Ri+99JLefvttSdLrr7+uqqoqff/735ck7dmzR01NTUpPT7de43A4NHLkSG3atEmSVFNTo/b2dr+My+VScnKyldm8ebMMw7AKkyQNGzZMhmH4ZZKTk63CJEkZGRny+Xyqqak56fh9Pp9aWlr8HgAA4NwU0N+emzVrlrxery6//HIFBwero6NDDz74oG655RZJUlNTkyQpNjbW73WxsbHau3evlQkNDVVUVFSXzInXNzU1KSYmpsv+Y2Ji/DKd9xMVFaXQ0FAr01lJSYnuu+++0z1sAADQBwV0punZZ5/V6tWr9fTTT2v79u168skntWjRIj355JN+uaCgIL/npml2WdZZ58zJ8t3JfNacOXPk9XqtR0NDwynHBAAA+q6AzjT97Gc/0+zZs3XzzTdLkoYMGaK9e/eqpKREkyZNktPplPTpLNCFF15ova65udmaFXI6nWpra5PH4/GbbWpubtbw4cOtzIEDB7rs/+DBg37b2bJli996j8ej9vb2LjNQJzgcDjkcju4ePgAA6EMCOtP00Ucf6bzz/IcQHBxs3XIgPj5eTqdTlZWV1vq2tjZt3LjRKkQpKSnq16+fX6axsVF1dXVWJi0tTV6vV1u3brUyW7Zskdfr9cvU1dWpsbHRylRUVMjhcCglJaWHjxwAAPQ1AZ1pGj9+vB588EFdfPHFuuKKK/Taa69p8eLFuuOOOyR9+nFZQUGBiouLlZCQoISEBBUXF6t///7KycmRJBmGoSlTpqiwsFADBw7UgAEDVFRUpCFDhmjMmDGSpMGDB2vcuHHKzc3VsmXLJElTp05VVlaWEhMTJUnp6elKSkqS2+3WwoULdejQIRUVFSk3N7fHvwkHAAD6noCWpkceeUT//d//rby8PDU3N8vlcmnatGn6f//v/1mZmTNnqrW1VXl5efJ4PEpNTVVFRYUiIiKszJIlSxQSEqIJEyaotbVVo0eP1sqVKxUcHGxl1qxZo/z8fOtbdtnZ2SotLbXWBwcHa/369crLy9OIESMUFhamnJwcLVq06Cy8EwAAoLcL6H2azjXcp6nv4z5NAPDl0yfu0wQAANBXUJoAAABsoDQBAADYQGkCAACwgdIEAABgA6UJAADABkoTAACADZQmAAAAGyhNAAAANlCaAAAAbKA0AQAA2EBpAgAAsIHSBAAAYAOlCQAAwAZKEwAAgA2UJgAAABsoTQAAADZQmgAAAGygNAEAANhAaQIAALCB0gQAAGADpQkAAMAGShMAAIANlCYAAAAbKE0AAAA2UJoAAABsoDQBAADYQGkCAACwgdIEAABgA6UJAADABkoTAACADQEtTZdeeqmCgoK6PO666y5Jkmmamjt3rlwul8LCwjRq1Cjt3LnTbxs+n0/Tp09XdHS0wsPDlZ2drX379vllPB6P3G63DMOQYRhyu906fPiwX6a+vl7jx49XeHi4oqOjlZ+fr7a2tjN6/AAAoO8IaGnatm2bGhsbrUdlZaUk6aabbpIkLViwQIsXL1Zpaam2bdsmp9OpsWPH6siRI9Y2CgoKtG7dOpWVlamqqkpHjx5VVlaWOjo6rExOTo5qa2tVXl6u8vJy1dbWyu12W+s7OjqUmZmpY8eOqaqqSmVlZVq7dq0KCwvP0jsBAAB6uyDTNM1AD+KEgoIC/fnPf9Y777wjSXK5XCooKNCsWbMkfTqrFBsbq/nz52vatGnyer0aNGiQVq1apYkTJ0qS9u/fr7i4OL3wwgvKyMjQrl27lJSUpOrqaqWmpkqSqqurlZaWprfeekuJiYnasGGDsrKy1NDQIJfLJUkqKyvT5MmT1dzcrMjISFvjb2lpkWEY8nq9tl9j16Wz1/fo9nBy783LDPQQAABnmd2/373mmqa2tjatXr1ad9xxh4KCgrRnzx41NTUpPT3dyjgcDo0cOVKbNm2SJNXU1Ki9vd0v43K5lJycbGU2b94swzCswiRJw4YNk2EYfpnk5GSrMElSRkaGfD6fampqPnfMPp9PLS0tfg8AAHBu6jWl6Y9//KMOHz6syZMnS5KampokSbGxsX652NhYa11TU5NCQ0MVFRV1ykxMTEyX/cXExPhlOu8nKipKoaGhVuZkSkpKrOukDMNQXFzcaRwxAADoS3pNaXriiSd03XXX+c32SFJQUJDfc9M0uyzrrHPmZPnuZDqbM2eOvF6v9WhoaDjluAAAQN/VK0rT3r179eKLL+rHP/6xtczpdEpSl5me5uZma1bI6XSqra1NHo/nlJkDBw502efBgwf9Mp334/F41N7e3mUG6rMcDociIyP9HgAA4NzUK0rTihUrFBMTo8zM/7sINz4+Xk6n0/pGnfTpdU8bN27U8OHDJUkpKSnq16+fX6axsVF1dXVWJi0tTV6vV1u3brUyW7Zskdfr9cvU1dWpsbHRylRUVMjhcCglJeXMHDQAAOhTQgI9gOPHj2vFihWaNGmSQkL+bzhBQUEqKChQcXGxEhISlJCQoOLiYvXv3185OTmSJMMwNGXKFBUWFmrgwIEaMGCAioqKNGTIEI0ZM0aSNHjwYI0bN065ublatmyZJGnq1KnKyspSYmKiJCk9PV1JSUlyu91auHChDh06pKKiIuXm5jJ7BAAAJPWC0vTiiy+qvr5ed9xxR5d1M2fOVGtrq/Ly8uTxeJSamqqKigpFRERYmSVLligkJEQTJkxQa2urRo8erZUrVyo4ONjKrFmzRvn5+da37LKzs1VaWmqtDw4O1vr165WXl6cRI0YoLCxMOTk5WrRo0Rk8cgAA0Jf0qvs09XXcp6nv4z5NAPDl0+fu0wQAANCbUZoAAABsoDQBAADYQGkCAACwgdIEAABgA6UJAADABkoTAACADZQmAAAAGyhNAAAANlCaAAAAbKA0AQAA2EBpAgAAsIHSBAAAYAOlCQAAwAZKEwAAgA2UJgAAABsoTQAAADZQmgAAAGygNAEAANhAaQIAALCB0gQAAGADpQkAAMAGShMAAIANlCYAAAAbKE0AAAA2UJoAAABsoDQBAADYQGkCAACwgdIEAABgA6UJAADABkoTAACADZQmAAAAGwJemt5//33ddtttGjhwoPr3769vfvObqqmpsdabpqm5c+fK5XIpLCxMo0aN0s6dO/224fP5NH36dEVHRys8PFzZ2dnat2+fX8bj8cjtdsswDBmGIbfbrcOHD/tl6uvrNX78eIWHhys6Olr5+flqa2s7Y8cOAAD6joCWJo/HoxEjRqhfv37asGGD3nzzTT300EO64IILrMyCBQu0ePFilZaWatu2bXI6nRo7dqyOHDliZQoKCrRu3TqVlZWpqqpKR48eVVZWljo6OqxMTk6OamtrVV5ervLyctXW1srtdlvrOzo6lJmZqWPHjqmqqkplZWVau3atCgsLz8p7AQAAercg0zTNQO189uzZ+sc//qG///3vJ11vmqZcLpcKCgo0a9YsSZ/OKsXGxmr+/PmaNm2avF6vBg0apFWrVmnixImSpP379ysuLk4vvPCCMjIytGvXLiUlJam6ulqpqamSpOrqaqWlpemtt95SYmKiNmzYoKysLDU0NMjlckmSysrKNHnyZDU3NysyMvILj6elpUWGYcjr9drKn45LZ6/v0e3h5N6blxnoIQAAzjK7f78DOtP0/PPPa+jQobrpppsUExOjq666SsuXL7fW79mzR01NTUpPT7eWORwOjRw5Ups2bZIk1dTUqL293S/jcrmUnJxsZTZv3izDMKzCJEnDhg2TYRh+meTkZKswSVJGRoZ8Pp/fx4Wf5fP51NLS4vcAAADnpoCWpnfffVdLly5VQkKC/vKXv+jOO+9Ufn6+nnrqKUlSU1OTJCk2NtbvdbGxsda6pqYmhYaGKioq6pSZmJiYLvuPiYnxy3TeT1RUlEJDQ61MZyUlJdY1UoZhKC4u7nTfAgAA0EcEtDQdP35cV199tYqLi3XVVVdp2rRpys3N1dKlS/1yQUFBfs9N0+yyrLPOmZPlu5P5rDlz5sjr9VqPhoaGU44JAAD0XQEtTRdeeKGSkpL8lg0ePFj19fWSJKfTKUldZnqam5utWSGn06m2tjZ5PJ5TZg4cONBl/wcPHvTLdN6Px+NRe3t7lxmoExwOhyIjI/0eAADg3BTQ0jRixAjt3r3bb9nbb7+tSy65RJIUHx8vp9OpyspKa31bW5s2btyo4cOHS5JSUlLUr18/v0xjY6Pq6uqsTFpamrxer7Zu3WpltmzZIq/X65epq6tTY2OjlamoqJDD4VBKSkoPHzkAAOhrQgK585/+9KcaPny4iouLNWHCBG3dulWPP/64Hn/8cUmfflxWUFCg4uJiJSQkKCEhQcXFxerfv79ycnIkSYZhaMqUKSosLNTAgQM1YMAAFRUVaciQIRozZoykT2evxo0bp9zcXC1btkySNHXqVGVlZSkxMVGSlJ6erqSkJLndbi1cuFCHDh1SUVGRcnNzmUECAACBLU3f+ta3tG7dOs2ZM0f333+/4uPj9fDDD+vWW2+1MjNnzlRra6vy8vLk8XiUmpqqiooKRUREWJklS5YoJCREEyZMUGtrq0aPHq2VK1cqODjYyqxZs0b5+fnWt+yys7NVWlpqrQ8ODtb69euVl5enESNGKCwsTDk5OVq0aNFZeCcAAEBvF9D7NJ1ruE9T38d9mgDgy6dP3KcJAACgr6A0AQAA2EBpAgAAsIHSBAAAYAOlCQAAwAZKEwAAgA2UJgAAABsoTQAAADZQmgAAAGygNAEAANhAaQIAALCB0gQAAGADpQkAAMAGShMAAIANlCYAAAAbKE0AAAA2UJoAAABsoDQBAADYQGkCAACwgdIEAABgA6UJAADABkoTAACADZQmAAAAGyhNAAAANlCaAAAAbKA0AQAA2EBpAgAAsIHSBAAAYAOlCQAAwAZKEwAAgA2UJgAAABu6VZr27NnTIzufO3eugoKC/B5Op9Nab5qm5s6dK5fLpbCwMI0aNUo7d+7024bP59P06dMVHR2t8PBwZWdna9++fX4Zj8cjt9stwzBkGIbcbrcOHz7sl6mvr9f48eMVHh6u6Oho5efnq62trUeOEwAA9H3dKk1f+9rXdO2112r16tX6+OOP/6MBXHHFFWpsbLQeO3bssNYtWLBAixcvVmlpqbZt2yan06mxY8fqyJEjVqagoEDr1q1TWVmZqqqqdPToUWVlZamjo8PK5OTkqLa2VuXl5SovL1dtba3cbre1vqOjQ5mZmTp27JiqqqpUVlamtWvXqrCw8D86NgAAcO7oVml6/fXXddVVV6mwsFBOp1PTpk3T1q1buzWAkJAQOZ1O6zFo0CBJn84yPfzww7r33nt14403Kjk5WU8++aQ++ugjPf3005Ikr9erJ554Qg899JDGjBmjq666SqtXr9aOHTv04osvSpJ27dql8vJy/fa3v1VaWprS0tK0fPly/fnPf9bu3bslSRUVFXrzzTe1evVqXXXVVRozZoweeughLV++XC0tLd06LgAAcG7pVmlKTk7W4sWL9f7772vFihVqamrSNddcoyuuuEKLFy/WwYMHbW/rnXfekcvlUnx8vG6++Wa9++67kj79CLCpqUnp6elW1uFwaOTIkdq0aZMkqaamRu3t7X4Zl8ul5ORkK7N582YZhqHU1FQrM2zYMBmG4ZdJTk6Wy+WyMhkZGfL5fKqpqenGOwQAAM41/9GF4CEhIbrhhhv0u9/9TvPnz9e//vUvFRUV6aKLLtLtt9+uxsbGU74+NTVVTz31lP7yl79o+fLlampq0vDhw/Xhhx+qqalJkhQbG+v3mtjYWGtdU1OTQkNDFRUVdcpMTExMl33HxMT4ZTrvJyoqSqGhoVbmZHw+n1paWvweAADg3PQflaZXX31VeXl5uvDCC7V48WIVFRXpX//6l15++WW9//77+sEPfnDK11933XX64Q9/qCFDhmjMmDFav369JOnJJ5+0MkFBQX6vMU2zy7LOOmdOlu9OprOSkhLr4nLDMBQXF3fKcQEAgL6rW6Vp8eLFGjJkiIYPH679+/frqaee0t69e/XAAw8oPj5eI0aM0LJly7R9+/bT2m54eLiGDBmid955x/oWXeeZnubmZmtWyOl0qq2tTR6P55SZAwcOdNnXwYMH/TKd9+PxeNTe3t5lBuqz5syZI6/Xaz0aGhpO63gBAEDf0a3StHTpUuXk5Ki+vl5//OMflZWVpfPO89/UxRdfrCeeeOK0tuvz+bRr1y5deOGFio+Pl9PpVGVlpbW+ra1NGzdu1PDhwyVJKSkp6tevn1+msbFRdXV1ViYtLU1er9fvQvUtW7bI6/X6Zerq6vw+TqyoqJDD4VBKSsrnjtfhcCgyMtLvAQAAzk0h3XnRO++884WZ0NBQTZo06ZSZoqIijR8/XhdffLGam5v1wAMPqKWlRZMmTVJQUJAKCgpUXFyshIQEJSQkqLi4WP3791dOTo4kyTAMTZkyRYWFhRo4cKAGDBigoqIi6+M+SRo8eLDGjRun3NxcLVu2TJI0depUZWVlKTExUZKUnp6upKQkud1uLVy4UIcOHVJRUZFyc3MpQgAAQFI3S9OKFSv0la98RTfddJPf8t///vf66KOPvrAsnbBv3z7dcsst+uCDDzRo0CANGzZM1dXVuuSSSyRJM2fOVGtrq/Ly8uTxeJSamqqKigpFRERY21iyZIlCQkI0YcIEtba2avTo0Vq5cqWCg4OtzJo1a5Sfn299yy47O1ulpaXW+uDgYK1fv155eXkaMWKEwsLClJOTo0WLFnXn7QEAAOegINM0zdN9UWJioh577DFde+21fss3btyoqVOnWvc/+rJpaWmRYRjyer09PkN16ez1Pbo9nNx78zIDPQQAwFlm9+93t65p2rt3r+Lj47ssv+SSS1RfX9+dTQIAAPRq3SpNMTExeuONN7osf/311zVw4MD/eFAAAAC9TbdK080336z8/Hy98sor6ujoUEdHh15++WXNmDFDN998c0+PEQAAIOC6dSH4Aw88oL1792r06NEKCfl0E8ePH9ftt9+u4uLiHh0gAABAb9Ct0hQaGqpnn31W//M//6PXX39dYWFhGjJkiPWtNwAAgHNNt0rTCV//+tf19a9/vafGAgAA0Gt1qzR1dHRo5cqVeumll9Tc3Kzjx4/7rX/55Zd7ZHAAAAC9RbdK04wZM7Ry5UplZmYqOTn5C39AFwAAoK/rVmkqKyvT7373O33/+9/v6fEAAAD0St265UBoaKi+9rWv9fRYAAAAeq1ulabCwkL96le/Ujd+gQUAAKBP6tbHc1VVVXrllVe0YcMGXXHFFerXr5/f+ueee65HBgcAANBbdKs0XXDBBbrhhht6eiwAAAC9VrdK04oVK3p6HAAAAL1at65pkqRPPvlEL774opYtW6YjR45Ikvbv36+jR4/22OAAAAB6i27NNO3du1fjxo1TfX29fD6fxo4dq4iICC1YsEAff/yxHnvssZ4eJwAAQEB1a6ZpxowZGjp0qDwej8LCwqzlN9xwg1566aUeGxwAAEBv0e1vz/3jH/9QaGio3/JLLrlE77//fo8MDAAAoDfp1kzT8ePH1dHR0WX5vn37FBER8R8PCgAAoLfpVmkaO3asHn74Yet5UFCQjh49ql/+8pf8tAoAADgndevjuSVLlujaa69VUlKSPv74Y+Xk5Oidd95RdHS0nnnmmZ4eIwAAQMB1qzS5XC7V1tbqmWee0fbt23X8+HFNmTJFt956q9+F4QAAAOeKbpUmSQoLC9Mdd9yhO+64oyfHAwAA0Ct1qzQ99dRTp1x/++23d2swAAAAvVW3StOMGTP8nre3t+ujjz5SaGio+vfvT2kCAADnnG59e87j8fg9jh49qt27d+uaa67hQnAAAHBO6vZvz3WWkJCgefPmdZmFAgAAOBf0WGmSpODgYO3fv78nNwkAANArdOuapueff97vuWmaamxsVGlpqUaMGNEjAwMAAOhNulWarr/+er/nQUFBGjRokL73ve/poYce6olxAQAA9CrdKk3Hjx/v6XEAAAD0aj16TRMAAMC5qlszTffcc4/t7OLFi23lSkpK9POf/1wzZsywfgzYNE3dd999evzxx+XxeJSamqrf/OY3uuKKK6zX+Xw+FRUV6ZlnnlFra6tGjx6tRx99VBdddJGV8Xg8ys/Pt67Fys7O1iOPPKILLrjAytTX1+uuu+7Syy+/rLCwMOXk5GjRokUKDQ21fawAAODc1a3S9Nprr2n79u365JNPlJiYKEl6++23FRwcrKuvvtrKBQUF2dretm3b9Pjjj+vKK6/0W75gwQItXrxYK1eu1Ne//nU98MADGjt2rHbv3q2IiAhJUkFBgf70pz+prKxMAwcOVGFhobKyslRTU6Pg4GBJUk5Ojvbt26fy8nJJ0tSpU+V2u/WnP/1JktTR0aHMzEwNGjRIVVVV+vDDDzVp0iSZpqlHHnmkO28RAAA4x3SrNI0fP14RERF68sknFRUVJenT2Zwf/ehH+s53vqPCwkLb2zp69KhuvfVWLV++XA888IC13DRNPfzww7r33nt14403SpKefPJJxcbG6umnn9a0adPk9Xr1xBNPaNWqVRozZowkafXq1YqLi9OLL76ojIwM7dq1S+Xl5aqurlZqaqokafny5UpLS9Pu3buVmJioiooKvfnmm2poaJDL5ZIkPfTQQ5o8ebIefPBBRUZGdudtAgAA55BuXdP00EMPqaSkxCpMkhQVFaUHHnjgtL89d9dddykzM9MqPSfs2bNHTU1NSk9Pt5Y5HA6NHDlSmzZtkiTV1NSovb3dL+NyuZScnGxlNm/eLMMwrMIkScOGDZNhGH6Z5ORkqzBJUkZGhnw+n2pqaj537D6fTy0tLX4PAABwbupWaWppadGBAwe6LG9ubtaRI0dsb6esrEzbt29XSUlJl3VNTU2SpNjYWL/lsbGx1rqmpiaFhob6lbeTZWJiYrpsPyYmxi/TeT9RUVEKDQ21MidTUlIiwzCsR1xc3BcdMgAA6KO6VZpuuOEG/ehHP9If/vAH7du3T/v27dMf/vAHTZkyxfoo7Ys0NDRoxowZWr16tc4///zPzXW+Lso0zS+8Vqpz5mT57mQ6mzNnjrxer/VoaGg45bgAAEDf1a1rmh577DEVFRXptttuU3t7+6cbCgnRlClTtHDhQlvbqKmpUXNzs1JSUqxlHR0d+tvf/qbS0lLt3r1b0qezQBdeeKGVaW5utmaFnE6n2tra5PF4/GabmpubNXz4cCtzslmxgwcP+m1ny5Ytfus9Ho/a29u7zEB9lsPhkMPhsHW8AACgb+vWTFP//v316KOP6sMPP7S+SXfo0CE9+uijCg8Pt7WN0aNHa8eOHaqtrbUeQ4cO1a233qra2lpddtllcjqdqqystF7T1tamjRs3WoUoJSVF/fr188s0Njaqrq7OyqSlpcnr9Wrr1q1WZsuWLfJ6vX6Zuro6NTY2WpmKigo5HA6/UgcAAL68ujXTdEJjY6MaGxv13e9+V2FhYbY+OjshIiJCycnJfsvCw8M1cOBAa3lBQYGKi4uVkJCghIQEFRcXq3///srJyZEkGYahKVOmqLCwUAMHDtSAAQNUVFSkIUOGWBeWDx48WOPGjVNubq6WLVsm6dNbDmRlZVm3S0hPT1dSUpLcbrcWLlyoQ4cOqaioSLm5uXxzDgAASOpmafrwww81YcIEvfLKKwoKCtI777yjyy67TD/+8Y91wQUX9Njvz82cOVOtra3Ky8uzbm5ZUVFh3aNJkpYsWaKQkBBNmDDBurnlypUrrXs0SdKaNWuUn59vfcsuOztbpaWl1vrg4GCtX79eeXl5GjFihN/NLQEAACQpyDRN83RfdPvtt6u5uVm//e1vNXjwYL3++uu67LLLVFFRoZ/+9KfauXPnmRhrr9fS0iLDMOT1ent8hurS2et7dHs4uffmZQZ6CACAs8zu3+9uzTRVVFToL3/5i99PlUhSQkKC9u7d251NAgAA9GrduhD82LFj6t+/f5flH3zwAd8mAwAA56Rulabvfve7euqpp6znQUFBOn78uBYuXKhrr722xwYHAADQW3Tr47mFCxdq1KhRevXVV9XW1qaZM2dq586dOnTokP7xj3/09BgBAAACrlszTUlJSXrjjTf07W9/W2PHjtWxY8d044036rXXXtN//dd/9fQYAQAAAu60Z5pO/EDusmXLdN99952JMQEAAPQ6pz3T1K9fP9XV1dm+iSUAAMC5oFsfz91+++164oknenosAAAAvVa3LgRva2vTb3/7W1VWVmro0KFdfm9u8eLFPTI4AACA3uK0StO7776rSy+9VHV1dbr66qslSW+//bZfho/tAADAuei0SlNCQoIaGxv1yiuvSJImTpyoX//614qNjT0jgwMAAOgtTuuaps4/U7dhwwYdO3asRwcEAADQG3XrQvATuvFbvwAAAH3SaZWmoKCgLtcscQ0TAAD4Mjita5pM09TkyZOtH+X9+OOPdeedd3b59txzzz3XcyMEAADoBU6rNE2aNMnv+W233dajgwEAAOitTqs0rVix4kyNAwAAoFf7jy4EBwAA+LKgNAEAANhAaQIAALCB0gQAAGADpQkAAMAGShMAAIANlCYAAAAbKE0AAAA2UJoAAABsoDQBAADYQGkCAACwgdIEAABgA6UJAADABkoTAACADQEtTUuXLtWVV16pyMhIRUZGKi0tTRs2bLDWm6apuXPnyuVyKSwsTKNGjdLOnTv9tuHz+TR9+nRFR0crPDxc2dnZ2rdvn1/G4/HI7XbLMAwZhiG3263Dhw/7Zerr6zV+/HiFh4crOjpa+fn5amtrO2PHDgAA+paAlqaLLrpI8+bN06uvvqpXX31V3/ve9/SDH/zAKkYLFizQ4sWLVVpaqm3btsnpdGrs2LE6cuSItY2CggKtW7dOZWVlqqqq0tGjR5WVlaWOjg4rk5OTo9raWpWXl6u8vFy1tbVyu93W+o6ODmVmZurYsWOqqqpSWVmZ1q5dq8LCwrP3ZgAAgF4tyDRNM9CD+KwBAwZo4cKFuuOOO+RyuVRQUKBZs2ZJ+nRWKTY2VvPnz9e0adPk9Xo1aNAgrVq1ShMnTpQk7d+/X3FxcXrhhReUkZGhXbt2KSkpSdXV1UpNTZUkVVdXKy0tTW+99ZYSExO1YcMGZWVlqaGhQS6XS5JUVlamyZMnq7m5WZGRkbbG3tLSIsMw5PV6bb/Grktnr+/R7eHk3puXGeghAADOMrt/v3vNNU0dHR0qKyvTsWPHlJaWpj179qipqUnp6elWxuFwaOTIkdq0aZMkqaamRu3t7X4Zl8ul5ORkK7N582YZhmEVJkkaNmyYDMPwyyQnJ1uFSZIyMjLk8/lUU1NzRo8bAAD0DSGBHsCOHTuUlpamjz/+WF/5yle0bt06JSUlWYUmNjbWLx8bG6u9e/dKkpqamhQaGqqoqKgumaamJisTExPTZb8xMTF+mc77iYqKUmhoqJU5GZ/PJ5/PZz1vaWmxe9gAAKCPCfhMU2Jiompra1VdXa2f/OQnmjRpkt58801rfVBQkF/eNM0uyzrrnDlZvjuZzkpKSqyLyw3DUFxc3CnHBQAA+q6Al6bQ0FB97Wtf09ChQ1VSUqJvfOMb+tWvfiWn0ylJXWZ6mpubrVkhp9OptrY2eTyeU2YOHDjQZb8HDx70y3Tej8fjUXt7e5cZqM+aM2eOvF6v9WhoaDjNowcAAH1FwEtTZ6ZpyufzKT4+Xk6nU5WVlda6trY2bdy4UcOHD5ckpaSkqF+/fn6ZxsZG1dXVWZm0tDR5vV5t3brVymzZskVer9cvU1dXp8bGRitTUVEhh8OhlJSUzx2rw+Gwbpdw4gEAAM5NAb2m6ec//7muu+46xcXF6ciRIyorK9Nf//pXlZeXKygoSAUFBSouLlZCQoISEhJUXFys/v37KycnR5JkGIamTJmiwsJCDRw4UAMGDFBRUZGGDBmiMWPGSJIGDx6scePGKTc3V8uWLZMkTZ06VVlZWUpMTJQkpaenKykpSW63WwsXLtShQ4dUVFSk3NxcihAAAJAU4NJ04MABud1uNTY2yjAMXXnllSovL9fYsWMlSTNnzlRra6vy8vLk8XiUmpqqiooKRUREWNtYsmSJQkJCNGHCBLW2tmr06NFauXKlgoODrcyaNWuUn59vfcsuOztbpaWl1vrg4GCtX79eeXl5GjFihMLCwpSTk6NFixadpXcCAAD0dr3uPk19Gfdp6vu4TxMAfPn0ufs0AQAA9GaUJgAAABsoTQAAADZQmgAAAGygNAEAANhAaQIAALCB0gQAAGADpQkAAMAGShMAAIANlCYAAAAbKE0AAAA2UJoAAABsoDQBAADYQGkCAACwgdIEAABgA6UJAADABkoTAACADZQmAAAAGyhNAAAANlCaAAAAbKA0AQAA2EBpAgAAsIHSBAAAYAOlCQAAwAZKEwAAgA2UJgAAABsoTQAAADZQmgAAAGygNAEAANhAaQIAALCB0gQAAGADpQkAAMCGgJamkpISfetb31JERIRiYmJ0/fXXa/fu3X4Z0zQ1d+5cuVwuhYWFadSoUdq5c6dfxufzafr06YqOjlZ4eLiys7O1b98+v4zH45Hb7ZZhGDIMQ263W4cPH/bL1NfXa/z48QoPD1d0dLTy8/PV1tZ2Ro4dAAD0LQEtTRs3btRdd92l6upqVVZW6pNPPlF6erqOHTtmZRYsWKDFixertLRU27Ztk9Pp1NixY3XkyBErU1BQoHXr1qmsrExVVVU6evSosrKy1NHRYWVycnJUW1ur8vJylZeXq7a2Vm6321rf0dGhzMxMHTt2TFVVVSorK9PatWtVWFh4dt4MAADQqwWZpmkGehAnHDx4UDExMdq4caO++93vyjRNuVwuFRQUaNasWZI+nVWKjY3V/PnzNW3aNHm9Xg0aNEirVq3SxIkTJUn79+9XXFycXnjhBWVkZGjXrl1KSkpSdXW1UlNTJUnV1dVKS0vTW2+9pcTERG3YsEFZWVlqaGiQy+WSJJWVlWny5Mlqbm5WZGTkF46/paVFhmHI6/Xayp+OS2ev79Ht4eTem5cZ6CEAAM4yu3+/e9U1TV6vV5I0YMAASdKePXvU1NSk9PR0K+NwODRy5Eht2rRJklRTU6P29na/jMvlUnJyspXZvHmzDMOwCpMkDRs2TIZh+GWSk5OtwiRJGRkZ8vl8qqmpOel4fT6fWlpa/B4AAODc1GtKk2mauueee3TNNdcoOTlZktTU1CRJio2N9cvGxsZa65qamhQaGqqoqKhTZmJiYrrsMyYmxi/TeT9RUVEKDQ21Mp2VlJRY10gZhqG4uLjTPWwAANBH9JrSdPfdd+uNN97QM88802VdUFCQ33PTNLss66xz5mT57mQ+a86cOfJ6vdajoaHhlGMCAAB9V68oTdOnT9fzzz+vV155RRdddJG13Ol0SlKXmZ7m5mZrVsjpdKqtrU0ej+eUmQMHDnTZ78GDB/0ynffj8XjU3t7eZQbqBIfDocjISL8HAAA4NwW0NJmmqbvvvlvPPfecXn75ZcXHx/utj4+Pl9PpVGVlpbWsra1NGzdu1PDhwyVJKSkp6tevn1+msbFRdXV1ViYtLU1er1dbt261Mlu2bJHX6/XL1NXVqbGx0cpUVFTI4XAoJSWl5w8eAAD0KSGB3Pldd92lp59+Wv/7v/+riIgIa6bHMAyFhYUpKChIBQUFKi4uVkJCghISElRcXKz+/fsrJyfHyk6ZMkWFhYUaOHCgBgwYoKKiIg0ZMkRjxoyRJA0ePFjjxo1Tbm6uli1bJkmaOnWqsrKylJiYKElKT09XUlKS3G63Fi5cqEOHDqmoqEi5ubnMIAEAgMCWpqVLl0qSRo0a5bd8xYoVmjx5siRp5syZam1tVV5enjwej1JTU1VRUaGIiAgrv2TJEoWEhGjChAlqbW3V6NGjtXLlSgUHB1uZNWvWKD8/3/qWXXZ2tkpLS631wcHBWr9+vfLy8jRixAiFhYUpJydHixYtOkNHDwAA+pJedZ+mvo77NPV93KcJAL58+uR9mgAAAHorShMAAIANlCYAAAAbKE0AAAA2UJoAAABsoDQBAADYQGkCAACwgdIEAABgA6UJAADABkoTAACADZQmAAAAGyhNAAAANlCaAAAAbKA0AQAA2EBpAgAAsIHSBAAAYAOlCQAAwAZKEwAAgA2UJgAAABsoTQAAADZQmgAAAGygNAEAANhAaQIAALCB0gQAAGADpQkAAMAGShMAAIANlCYAAAAbKE0AAAA2UJoAAABsoDQBAADYQGkCAACwgdIEAABgQ0BL09/+9jeNHz9eLpdLQUFB+uMf/+i33jRNzZ07Vy6XS2FhYRo1apR27tzpl/H5fJo+fbqio6MVHh6u7Oxs7du3zy/j8XjkdrtlGIYMw5Db7dbhw4f9MvX19Ro/frzCw8MVHR2t/Px8tbW1nYnDBgAAfVBAS9OxY8f0jW98Q6WlpSddv2DBAi1evFilpaXatm2bnE6nxo4dqyNHjliZgoICrVu3TmVlZaqqqtLRo0eVlZWljo4OK5OTk6Pa2lqVl5ervLxctbW1crvd1vqOjg5lZmbq2LFjqqqqUllZmdauXavCwsIzd/AAAKBPCTJN0wz0ICQpKChI69at0/XXXy/p01kml8ulgoICzZo1S9Kns0qxsbGaP3++pk2bJq/Xq0GDBmnVqlWaOHGiJGn//v2Ki4vTCy+8oIyMDO3atUtJSUmqrq5WamqqJKm6ulppaWl66623lJiYqA0bNigrK0sNDQ1yuVySpLKyMk2ePFnNzc2KjIy0dQwtLS0yDENer9f2a+y6dPb6Ht0eTu69eZmBHgIA4Cyz+/e7117TtGfPHjU1NSk9Pd1a5nA4NHLkSG3atEmSVFNTo/b2dr+My+VScnKyldm8ebMMw7AKkyQNGzZMhmH4ZZKTk63CJEkZGRny+Xyqqan53DH6fD61tLT4PQAAwLmp15ampqYmSVJsbKzf8tjYWGtdU1OTQkNDFRUVdcpMTExMl+3HxMT4ZTrvJyoqSqGhoVbmZEpKSqzrpAzDUFxc3GkeJQAA6Ct6bWk6ISgoyO+5aZpdlnXWOXOyfHcync2ZM0der9d6NDQ0nHJcAACg7+q1pcnpdEpSl5me5uZma1bI6XSqra1NHo/nlJkDBw502f7Bgwf9Mp334/F41N7e3mUG6rMcDociIyP9HgAA4NzUa0tTfHy8nE6nKisrrWVtbW3auHGjhg8fLklKSUlRv379/DKNjY2qq6uzMmlpafJ6vdq6dauV2bJli7xer1+mrq5OjY2NVqaiokIOh0MpKSln9DgBAEDfEBLInR89elT//Oc/red79uxRbW2tBgwYoIsvvlgFBQUqLi5WQkKCEhISVFxcrP79+ysnJ0eSZBiGpkyZosLCQg0cOFADBgxQUVGRhgwZojFjxkiSBg8erHHjxik3N1fLli2TJE2dOlVZWVlKTEyUJKWnpyspKUlut1sLFy7UoUOHVFRUpNzcXGaPAACApACXpldffVXXXnut9fyee+6RJE2aNEkrV67UzJkz1draqry8PHk8HqWmpqqiokIRERHWa5YsWaKQkBBNmDBBra2tGj16tFauXKng4GArs2bNGuXn51vfssvOzva7N1RwcLDWr1+vvLw8jRgxQmFhYcrJydGiRYvO9FuAXoZbO5w93N4BQF/Ta+7TdC7gPk2AfZQmAL1Fn79PEwAAQG9CaQIAALCB0gQAAGADpQkAAMAGShMAAIANlCYAAAAbKE0AAAA2UJoAAABsoDQBAADYQGkCAACwgdIEAABgA6UJAADABkoTAACADZQmAAAAGyhNAAAANlCaAAAAbKA0AQAA2EBpAgAAsIHSBAAAYAOlCQAAwAZKEwAAgA0hgR4AgC+nS2evD/QQvhTem5cZ6CEA5wxmmgAAAGygNAEAANhAaQIAALCB0gQAAGADpQkAAMAGShMAAIANlCYAAAAbKE0AAAA2UJoAAABsoDR18uijjyo+Pl7nn3++UlJS9Pe//z3QQwIAAL0Apekznn32WRUUFOjee+/Va6+9pu985zu67rrrVF9fH+ihAQCAAAsyTdMM9CB6i9TUVF199dVaunSptWzw4MG6/vrrVVJS8oWvb2lpkWEY8nq9ioyM7NGx8TtdANB78Rt/fZvdv9/8YO+/tbW1qaamRrNnz/Zbnp6erk2bNp30NT6fTz6fz3ru9Xolffrm97Tjvo96fJsAgJ5xJv67j7PnxPn7onkkStO/ffDBB+ro6FBsbKzf8tjYWDU1NZ30NSUlJbrvvvu6LI+LizsjYwQA9E7Gw4EeAXrCkSNHZBjG566nNHUSFBTk99w0zS7LTpgzZ47uuece6/nx48d16NAhDRw48HNf0x0tLS2Ki4tTQ0NDj3/sh9PDueg9OBe9B+ei9+BcdI9pmjpy5IhcLtcpc5Smf4uOjlZwcHCXWaXm5uYus08nOBwOORwOv2UXXHDBmRqiIiMj+UfQS3Aueg/ORe/Bueg9OBen71QzTCfw7bl/Cw0NVUpKiiorK/2WV1ZWavjw4QEaFQAA6C2YafqMe+65R263W0OHDlVaWpoef/xx1dfX68477wz00AAAQIBRmj5j4sSJ+vDDD3X//fersbFRycnJeuGFF3TJJZcEdFwOh0O//OUvu3wUiLOPc9F7cC56D85F78G5OLO4TxMAAIANXNMEAABgA6UJAADABkoTAACADZQmAAAAGyhNfcCjjz6q+Ph4nX/++UpJSdHf//73QA+pzyopKdG3vvUtRUREKCYmRtdff712797tlzFNU3PnzpXL5VJYWJhGjRqlnTt3+mV8Pp+mT5+u6OhohYeHKzs7W/v27fPLeDweud1uGYYhwzDkdrt1+PDhM32IfVZJSYmCgoJUUFBgLeNcnD3vv/++brvtNg0cOFD9+/fXN7/5TdXU1FjrORdnxyeffKJf/OIXio+PV1hYmC677DLdf//9On78uJXhXASQiV6trKzM7Nevn7l8+XLzzTffNGfMmGGGh4ebe/fuDfTQ+qSMjAxzxYoVZl1dnVlbW2tmZmaaF198sXn06FErM2/ePDMiIsJcu3atuWPHDnPixInmhRdeaLa0tFiZO++80/zqV79qVlZWmtu3bzevvfZa8xvf+Ib5ySefWJlx48aZycnJ5qZNm8xNmzaZycnJZlZW1lk93r5i69at5qWXXmpeeeWV5owZM6zlnIuz49ChQ+Yll1xiTp482dyyZYu5Z88e88UXXzT/+c9/WhnOxdnxwAMPmAMHDjT//Oc/m3v27DF///vfm1/5ylfMhx9+2MpwLgKH0tTLffvb3zbvvPNOv2WXX365OXv27ACN6NzS3NxsSjI3btxomqZpHj9+3HQ6nea8efOszMcff2wahmE+9thjpmma5uHDh81+/fqZZWVlVub99983zzvvPLO8vNw0TdN88803TUlmdXW1ldm8ebMpyXzrrbfOxqH1GUeOHDETEhLMyspKc+TIkVZp4lycPbNmzTKvueaaz13PuTh7MjMzzTvuuMNv2Y033mjedtttpmlyLgKNj+d6sba2NtXU1Cg9Pd1veXp6ujZt2hSgUZ1bvF6vJGnAgAGSpD179qipqcnvPXc4HBo5cqT1ntfU1Ki9vd0v43K5lJycbGU2b94swzCUmppqZYYNGybDMDh3ndx1113KzMzUmDFj/JZzLs6e559/XkOHDtVNN92kmJgYXXXVVVq+fLm1nnNx9lxzzTV66aWX9Pbbb0uSXn/9dVVVVen73/++JM5FoHFH8F7sgw8+UEdHR5cfDI6Nje3yw8I4faZp6p577tE111yj5ORkSbLe15O953v37rUyoaGhioqK6pI58fqmpibFxMR02WdMTAzn7jPKysq0fft2bdu2rcs6zsXZ8+6772rp0qW655579POf/1xbt25Vfn6+HA6Hbr/9ds7FWTRr1ix5vV5dfvnlCg4OVkdHhx588EHdcsstkvh3EWiUpj4gKCjI77lpml2W4fTdfffdeuONN1RVVdVlXXfe886Zk+U5d/+noaFBM2bMUEVFhc4///zPzXEuzrzjx49r6NChKi4uliRdddVV2rlzp5YuXarbb7/dynEuzrxnn31Wq1ev1tNPP60rrrhCtbW1KigokMvl0qRJk6wc5yIw+HiuF4uOjlZwcHCX1t/c3Nzl/zJweqZPn67nn39er7zyii666CJrudPplKRTvudOp1NtbW3yeDynzBw4cKDLfg8ePMi5+7eamho1NzcrJSVFISEhCgkJ0caNG/XrX/9aISEh1vvEuTjzLrzwQiUlJfktGzx4sOrr6yXx7+Js+tnPfqbZs2fr5ptv1pAhQ+R2u/XTn/5UJSUlkjgXgUZp6sVCQ0OVkpKiyspKv+WVlZUaPnx4gEbVt5mmqbvvvlvPPfecXn75ZcXHx/utj4+Pl9Pp9HvP29ratHHjRus9T0lJUb9+/fwyjY2NqqurszJpaWnyer3aunWrldmyZYu8Xi/n7t9Gjx6tHTt2qLa21noMHTpUt956q2pra3XZZZdxLs6SESNGdLn1xttvv239WDn/Ls6ejz76SOed5/+nOTg42LrlAOciwAJw8TlOw4lbDjzxxBPmm2++aRYUFJjh4eHme++9F+ih9Uk/+clPTMMwzL/+9a9mY2Oj9fjoo4+szLx580zDMMznnnvO3LFjh3nLLbec9Ou8F110kfniiy+a27dvN7/3ve+d9Ou8V155pbl582Zz8+bN5pAhQ/g67xf47LfnTJNzcbZs3brVDAkJMR988EHznXfeMdesWWP279/fXL16tZXhXJwdkyZNMr/61a9atxx47rnnzOjoaHPmzJlWhnMROJSmPuA3v/mNeckll5ihoaHm1VdfbX09HqdP0kkfK1assDLHjx83f/nLX5pOp9N0OBzmd7/7XXPHjh1+22ltbTXvvvtuc8CAAWZYWJiZlZVl1tfX+2U+/PBD89ZbbzUjIiLMiIgI89ZbbzU9Hs9ZOMq+q3Np4lycPX/605/M5ORk0+FwmJdffrn5+OOP+63nXJwdLS0t5owZM8yLL77YPP/8883LLrvMvPfee02fz2dlOBeBE2SaphnImS4AAIC+gGuaAAAAbKA0AQAA2EBpAgAAsIHSBAAAYAOlCQAAwAZKEwAAgA2UJgAAABsoTQAAADZQmgAAAGygNAEAANhAaQIAALCB0gQAAGDD/wegoahyaqmc8gAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Age upon Outcome Days\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAk0AAAGdCAYAAAAPLEfqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAAA7pElEQVR4nO3df1yV9cH/8TeBnJDBFYpwOouK7jGSsK1wQ7Q7bSrYQFbtkRZ10uXQRoksmD/W7u+su8BfaVssM9dDSy3aZu5uMxn0Y25MUcMoMbO2TDBBLI8HNToQXt8/ur3uDphdMvSAvZ6Px/njXNf7XNfnOtfDePc517lOkGmapgAAAHBK5wV6AAAAAH0BpQkAAMAGShMAAIANlCYAAAAbKE0AAAA2UJoAAABsoDQBAADYQGkCAACwISTQAziXHD9+XPv371dERISCgoICPRwAAGCDaZo6cuSIXC6Xzjvvi+eTKE09aP/+/YqLiwv0MAAAQDc0NDTooosu+sL1lKYeFBERIemzNz0yMjLAowEAAHa0tLQoLi7O+jv+RShNPejER3KRkZGUJgAA+pgvu7SGC8EBAABsoDQBAADYQGkCAACwgdIEAABgA6UJAADABkoTAACADZQmAAAAGyhNAAAANlCaAAAAbKA0AQAA2EBpAgAAsIHSBAAAYAOlCQAAwAZKEwAAgA0hgR4A7Ll09vpAD+Er4f15mYEeAgCgl2KmCQAAwAZKEwAAgA2UJgAAABsoTQAAADZQmgAAAGygNAEAANgQ0NL06aef6he/+IXi4+MVFhamyy67TA888ICOHz9uZUzT1Ny5c+VyuRQWFqZRo0Zp586dftvx+XyaPn26oqOjFR4eruzsbO3bt88v4/F45Ha7ZRiGDMOQ2+3W4cOH/TL19fUaP368wsPDFR0drfz8fLW1tZ2x4wcAAH1HQEvT/Pnz9fjjj6u0tFS7du3SggULtHDhQj366KNWZsGCBVq8eLFKS0u1bds2OZ1OjR07VkeOHLEyBQUFWrduncrKylRVVaWjR48qKytLHR0dViYnJ0e1tbUqLy9XeXm5amtr5Xa7rfUdHR3KzMzUsWPHVFVVpbKyMq1du1aFhYVn580AAAC9WpBpmmagdp6VlaXY2Fg9+eST1rIf/vCH6t+/v1atWiXTNOVyuVRQUKBZs2ZJ+mxWKTY2VvPnz9e0adPk9Xo1aNAgrVq1ShMnTpQk7d+/X3FxcXrxxReVkZGhXbt2KSkpSdXV1UpNTZUkVVdXKy0tTW+//bYSExO1YcMGZWVlqaGhQS6XS5JUVlamyZMnq7m5WZGRkV96PC0tLTIMQ16v11b+dHBzy7ODm1sCwFeP3b/fAZ1puuaaa/Tyyy/rnXfekSS98cYbqqqq0ve//31J0p49e9TU1KT09HTrNQ6HQyNHjtSmTZskSTU1NWpvb/fLuFwuJScnW5nNmzfLMAyrMEnSsGHDZBiGXyY5OdkqTJKUkZEhn8+nmpqak47f5/OppaXF7wEAAM5NAf0ZlVmzZsnr9eryyy9XcHCwOjo69NBDD+nWW2+VJDU1NUmSYmNj/V4XGxurvXv3WpnQ0FBFRUV1yZx4fVNTk2JiYrrsPyYmxi/TeT9RUVEKDQ21Mp2VlJTo/vvvP93DBgAAfVBAZ5qee+45rV69Ws8884y2b9+up556SosWLdJTTz3llwsKCvJ7bppml2Wddc6cLN+dzOfNmTNHXq/XejQ0NJxyTAAAoO8K6EzTz372M82ePVu33HKLJGnIkCHau3evSkpKNGnSJDmdTkmfzQJdeOGF1uuam5utWSGn06m2tjZ5PB6/2abm5mYNHz7cyhw4cKDL/g8ePOi3nS1btvit93g8am9v7zIDdYLD4ZDD4eju4QMAgD4koDNNH3/8sc47z38IwcHB1i0H4uPj5XQ6VVlZaa1va2vTxo0brUKUkpKifv36+WUaGxtVV1dnZdLS0uT1erV161Yrs2XLFnm9Xr9MXV2dGhsbrUxFRYUcDodSUlJ6+MgBAEBfE9CZpvHjx+uhhx7SxRdfrCuuuEKvv/66Fi9erDvvvFPSZx+XFRQUqLi4WAkJCUpISFBxcbH69++vnJwcSZJhGJoyZYoKCws1cOBADRgwQEVFRRoyZIjGjBkjSRo8eLDGjRun3NxcLVu2TJI0depUZWVlKTExUZKUnp6upKQkud1uLVy4UIcOHVJRUZFyc3N7/JtwAACg7wloaXr00Uf1X//1X8rLy1Nzc7NcLpemTZum//f//p+VmTlzplpbW5WXlyePx6PU1FRVVFQoIiLCyixZskQhISGaMGGCWltbNXr0aK1cuVLBwcFWZs2aNcrPz7e+ZZedna3S0lJrfXBwsNavX6+8vDyNGDFCYWFhysnJ0aJFi87COwEAAHq7gN6n6VzDfZr6Pu7TBABfPX3iPk0AAAB9BaUJAADABkoTAACADZQmAAAAGyhNAAAANlCaAAAAbKA0AQAA2EBpAgAAsIHSBAAAYAOlCQAAwAZKEwAAgA2UJgAAABsoTQAAADZQmgAAAGygNAEAANhAaQIAALCB0gQAAGADpQkAAMAGShMAAIANlCYAAAAbKE0AAAA2UJoAAABsoDQBAADYQGkCAACwgdIEAABgA6UJAADABkoTAACADZQmAAAAGyhNAAAANlCaAAAAbKA0AQAA2EBpAgAAsCGgpenSSy9VUFBQl8fdd98tSTJNU3PnzpXL5VJYWJhGjRqlnTt3+m3D5/Np+vTpio6OVnh4uLKzs7Vv3z6/jMfjkdvtlmEYMgxDbrdbhw8f9svU19dr/PjxCg8PV3R0tPLz89XW1nZGjx8AAPQdAS1N27ZtU2Njo/WorKyUJN18882SpAULFmjx4sUqLS3Vtm3b5HQ6NXbsWB05csTaRkFBgdatW6eysjJVVVXp6NGjysrKUkdHh5XJyclRbW2tysvLVV5ertraWrndbmt9R0eHMjMzdezYMVVVVamsrExr165VYWHhWXonAABAbxdkmqYZ6EGcUFBQoD//+c969913JUkul0sFBQWaNWuWpM9mlWJjYzV//nxNmzZNXq9XgwYN0qpVqzRx4kRJ0v79+xUXF6cXX3xRGRkZ2rVrl5KSklRdXa3U1FRJUnV1tdLS0vT2228rMTFRGzZsUFZWlhoaGuRyuSRJZWVlmjx5spqbmxUZGWlr/C0tLTIMQ16v1/Zr7Lp09voe3R5O7v15mYEeAgDgLLP797vXXNPU1tam1atX684771RQUJD27NmjpqYmpaenWxmHw6GRI0dq06ZNkqSamhq1t7f7ZVwul5KTk63M5s2bZRiGVZgkadiwYTIMwy+TnJxsFSZJysjIkM/nU01NzReO2efzqaWlxe8BAADOTb2mNP3xj3/U4cOHNXnyZElSU1OTJCk2NtYvFxsba61rampSaGiooqKiTpmJiYnpsr+YmBi/TOf9REVFKTQ01MqcTElJiXWdlGEYiouLO40jBgAAfUmvKU1PPvmkrr/+er/ZHkkKCgrye26aZpdlnXXOnCzfnUxnc+bMkdfrtR4NDQ2nHBcAAOi7ekVp2rt3r1566SX9+Mc/tpY5nU5J6jLT09zcbM0KOZ1OtbW1yePxnDJz4MCBLvs8ePCgX6bzfjwej9rb27vMQH2ew+FQZGSk3wMAAJybekVpWrFihWJiYpSZ+X8X4cbHx8vpdFrfqJM+u+5p48aNGj58uCQpJSVF/fr188s0Njaqrq7OyqSlpcnr9Wrr1q1WZsuWLfJ6vX6Zuro6NTY2WpmKigo5HA6lpKScmYMGAAB9SkigB3D8+HGtWLFCkyZNUkjI/w0nKChIBQUFKi4uVkJCghISElRcXKz+/fsrJydHkmQYhqZMmaLCwkINHDhQAwYMUFFRkYYMGaIxY8ZIkgYPHqxx48YpNzdXy5YtkyRNnTpVWVlZSkxMlCSlp6crKSlJbrdbCxcu1KFDh1RUVKTc3FxmjwAAgKReUJpeeukl1dfX68477+yybubMmWptbVVeXp48Ho9SU1NVUVGhiIgIK7NkyRKFhIRowoQJam1t1ejRo7Vy5UoFBwdbmTVr1ig/P9/6ll12drZKS0ut9cHBwVq/fr3y8vI0YsQIhYWFKScnR4sWLTqDRw4AAPqSXnWfpr6O+zT1fdynCQC+evrcfZoAAAB6M0oTAACADZQmAAAAGyhNAAAANlCaAAAAbKA0AQAA2EBpAgAAsIHSBAAAYAOlCQAAwAZKEwAAgA2UJgAAABsoTQAAADZQmgAAAGygNAEAANhAaQIAALCB0gQAAGADpQkAAMAGShMAAIANlCYAAAAbKE0AAAA2UJoAAABsoDQBAADYQGkCAACwgdIEAABgA6UJAADABkoTAACADZQmAAAAGyhNAAAANlCaAAAAbKA0AQAA2EBpAgAAsCHgpemDDz7Q7bffroEDB6p///769re/rZqaGmu9aZqaO3euXC6XwsLCNGrUKO3cudNvGz6fT9OnT1d0dLTCw8OVnZ2tffv2+WU8Ho/cbrcMw5BhGHK73Tp8+LBfpr6+XuPHj1d4eLiio6OVn5+vtra2M3bsAACg7whoafJ4PBoxYoT69eunDRs26K233tLDDz+sCy64wMosWLBAixcvVmlpqbZt2yan06mxY8fqyJEjVqagoEDr1q1TWVmZqqqqdPToUWVlZamjo8PK5OTkqLa2VuXl5SovL1dtba3cbre1vqOjQ5mZmTp27JiqqqpUVlamtWvXqrCw8Ky8FwAAoHcLMk3TDNTOZ8+erX/84x/6+9//ftL1pmnK5XKpoKBAs2bNkvTZrFJsbKzmz5+vadOmyev1atCgQVq1apUmTpwoSdq/f7/i4uL04osvKiMjQ7t27VJSUpKqq6uVmpoqSaqurlZaWprefvttJSYmasOGDcrKylJDQ4NcLpckqaysTJMnT1Zzc7MiIyO/9HhaWlpkGIa8Xq+t/Om4dPb6Ht0eTu79eZmBHgIA4Cyz+/c7oDNNL7zwgoYOHaqbb75ZMTExuuqqq7R8+XJr/Z49e9TU1KT09HRrmcPh0MiRI7Vp0yZJUk1Njdrb2/0yLpdLycnJVmbz5s0yDMMqTJI0bNgwGYbhl0lOTrYKkyRlZGTI5/P5fVz4eT6fTy0tLX4PAABwbgpoaXrvvfe0dOlSJSQk6C9/+Yvuuusu5efn6+mnn5YkNTU1SZJiY2P9XhcbG2uta2pqUmhoqKKiok6ZiYmJ6bL/mJgYv0zn/URFRSk0NNTKdFZSUmJdI2UYhuLi4k73LQAAAH1EQEvT8ePHdfXVV6u4uFhXXXWVpk2bptzcXC1dutQvFxQU5PfcNM0uyzrrnDlZvjuZz5szZ468Xq/1aGhoOOWYAABA3xXQ0nThhRcqKSnJb9ngwYNVX18vSXI6nZLUZaanubnZmhVyOp1qa2uTx+M5ZebAgQNd9n/w4EG/TOf9eDwetbe3d5mBOsHhcCgyMtLvAQAAzk0BLU0jRozQ7t27/Za98847uuSSSyRJ8fHxcjqdqqystNa3tbVp48aNGj58uCQpJSVF/fr188s0Njaqrq7OyqSlpcnr9Wrr1q1WZsuWLfJ6vX6Zuro6NTY2WpmKigo5HA6lpKT08JEDAIC+JiSQO//pT3+q4cOHq7i4WBMmTNDWrVv1xBNP6IknnpD02cdlBQUFKi4uVkJCghISElRcXKz+/fsrJydHkmQYhqZMmaLCwkINHDhQAwYMUFFRkYYMGaIxY8ZI+mz2aty4ccrNzdWyZcskSVOnTlVWVpYSExMlSenp6UpKSpLb7dbChQt16NAhFRUVKTc3lxkkAAAQ2NL0ne98R+vWrdOcOXP0wAMPKD4+Xo888ohuu+02KzNz5ky1trYqLy9PHo9HqampqqioUEREhJVZsmSJQkJCNGHCBLW2tmr06NFauXKlgoODrcyaNWuUn59vfcsuOztbpaWl1vrg4GCtX79eeXl5GjFihMLCwpSTk6NFixadhXcCAAD0dgG9T9O5hvs09X3cpwkAvnr6xH2aAAAA+gpKEwAAgA2UJgAAABsoTQAAADZQmgAAAGygNAEAANhAaQIAALCB0gQAAGADpQkAAMAGShMAAIANlCYAAAAbKE0AAAA2UJoAAABsoDQBAADYQGkCAACwgdIEAABgA6UJAADABkoTAACADZQmAAAAGyhNAAAANlCaAAAAbKA0AQAA2EBpAgAAsIHSBAAAYAOlCQAAwAZKEwAAgA2UJgAAABsoTQAAADZQmgAAAGygNAEAANhAaQIAALCB0gQAAGBDt0rTnj17emTnc+fOVVBQkN/D6XRa603T1Ny5c+VyuRQWFqZRo0Zp586dftvw+XyaPn26oqOjFR4eruzsbO3bt88v4/F45Ha7ZRiGDMOQ2+3W4cOH/TL19fUaP368wsPDFR0drfz8fLW1tfXIcQIAgL6vW6XpG9/4hq677jqtXr1an3zyyb81gCuuuEKNjY3WY8eOHda6BQsWaPHixSotLdW2bdvkdDo1duxYHTlyxMoUFBRo3bp1KisrU1VVlY4ePaqsrCx1dHRYmZycHNXW1qq8vFzl5eWqra2V2+221nd0dCgzM1PHjh1TVVWVysrKtHbtWhUWFv5bxwYAAM4d3SpNb7zxhq666ioVFhbK6XRq2rRp2rp1a7cGEBISIqfTaT0GDRok6bNZpkceeUT33XefbrrpJiUnJ+upp57Sxx9/rGeeeUaS5PV69eSTT+rhhx/WmDFjdNVVV2n16tXasWOHXnrpJUnSrl27VF5ert/+9rdKS0tTWlqali9frj//+c/avXu3JKmiokJvvfWWVq9erauuukpjxozRww8/rOXLl6ulpaVbxwUAAM4t3SpNycnJWrx4sT744AOtWLFCTU1Nuuaaa3TFFVdo8eLFOnjwoO1tvfvuu3K5XIqPj9ctt9yi9957T9JnHwE2NTUpPT3dyjocDo0cOVKbNm2SJNXU1Ki9vd0v43K5lJycbGU2b94swzCUmppqZYYNGybDMPwyycnJcrlcViYjI0M+n081NTVfOHafz6eWlha/BwAAODf9WxeCh4SE6MYbb9Tvfvc7zZ8/X//6179UVFSkiy66SHfccYcaGxtP+frU1FQ9/fTT+stf/qLly5erqalJw4cP10cffaSmpiZJUmxsrN9rYmNjrXVNTU0KDQ1VVFTUKTMxMTFd9h0TE+OX6byfqKgohYaGWpmTKSkpsa6TMgxDcXFxpzxeAADQd/1bpem1115TXl6eLrzwQi1evFhFRUX617/+pVdeeUUffPCBfvCDH5zy9ddff71++MMfasiQIRozZozWr18vSXrqqaesTFBQkN9rTNPssqyzzpmT5buT6WzOnDnyer3Wo6Gh4ZTjAgAAfVe3StPixYs1ZMgQDR8+XPv379fTTz+tvXv36sEHH1R8fLxGjBihZcuWafv27ae13fDwcA0ZMkTvvvuu9S26zjM9zc3N1qyQ0+lUW1ubPB7PKTMHDhzosq+DBw/6ZTrvx+PxqL29vcsM1Oc5HA5FRkb6PQAAwLmpW6Vp6dKlysnJUX19vf74xz8qKytL553nv6mLL75YTz755Glt1+fzadeuXbrwwgsVHx8vp9OpyspKa31bW5s2btyo4cOHS5JSUlLUr18/v0xjY6Pq6uqsTFpamrxer9+F6lu2bJHX6/XL1NXV+X2cWFFRIYfDoZSUlNM6BgAAcG4K6c6L3n333S/NhIaGatKkSafMFBUVafz48br44ovV3NysBx98UC0tLZo0aZKCgoJUUFCg4uJiJSQkKCEhQcXFxerfv79ycnIkSYZhaMqUKSosLNTAgQM1YMAAFRUVWR/3SdLgwYM1btw45ebmatmyZZKkqVOnKisrS4mJiZKk9PR0JSUlye12a+HChTp06JCKioqUm5vL7BEAAJDUzdK0YsUKfe1rX9PNN9/st/z3v/+9Pv744y8tSyfs27dPt956qz788EMNGjRIw4YNU3V1tS655BJJ0syZM9Xa2qq8vDx5PB6lpqaqoqJCERER1jaWLFmikJAQTZgwQa2trRo9erRWrlyp4OBgK7NmzRrl5+db37LLzs5WaWmptT44OFjr169XXl6eRowYobCwMOXk5GjRokXdeXsAAMA5KMg0TfN0X5SYmKjHH39c1113nd/yjRs3aurUqdb9j75qWlpaZBiGvF5vj89QXTp7fY9uDyf3/rzMQA8BAHCW2f373a1rmvbu3av4+Pguyy+55BLV19d3Z5MAAAC9WrdKU0xMjN58880uy9944w0NHDjw3x4UAABAb9Ot0nTLLbcoPz9fr776qjo6OtTR0aFXXnlFM2bM0C233NLTYwQAAAi4bl0I/uCDD2rv3r0aPXq0QkI+28Tx48d1xx13qLi4uEcHCAAA0Bt0qzSFhobqueee03//93/rjTfeUFhYmIYMGWJ96w0AAOBc063SdMI3v/lNffOb3+ypsQAAAPRa3SpNHR0dWrlypV5++WU1Nzfr+PHjfutfeeWVHhkcAABAb9Gt0jRjxgytXLlSmZmZSk5O/tIf0AUAAOjrulWaysrK9Lvf/U7f//73e3o8AAAAvVK3bjkQGhqqb3zjGz09FgAAgF6rW6WpsLBQv/rVr9SNX2ABAADok7r18VxVVZVeffVVbdiwQVdccYX69evnt/7555/vkcEBAAD0Ft0qTRdccIFuvPHGnh4LAABAr9Wt0rRixYqeHgcAAECv1q1rmiTp008/1UsvvaRly5bpyJEjkqT9+/fr6NGjPTY4AACA3qJbM0179+7VuHHjVF9fL5/Pp7FjxyoiIkILFizQJ598oscff7ynxwkAABBQ3ZppmjFjhoYOHSqPx6OwsDBr+Y033qiXX365xwYHAADQW3T723P/+Mc/FBoa6rf8kksu0QcffNAjAwMAAOhNujXTdPz4cXV0dHRZvm/fPkVERPzbgwIAAOhtulWaxo4dq0ceecR6HhQUpKNHj+qXv/wlP60CAADOSd36eG7JkiW67rrrlJSUpE8++UQ5OTl69913FR0drWeffbanxwgAABBw3SpNLpdLtbW1evbZZ7V9+3YdP35cU6ZM0W233eZ3YTgAAMC5olulSZLCwsJ055136s477+zJ8QAAAPRK3SpNTz/99CnX33HHHd0aDAAAQG/VrdI0Y8YMv+ft7e36+OOPFRoaqv79+1OaAADAOadb357zeDx+j6NHj2r37t265ppruBAcAACck7r923OdJSQkaN68eV1moQAAAM4FPVaaJCk4OFj79+/vyU0CAAD0Ct26pumFF17we26aphobG1VaWqoRI0b0yMAAAAB6k26VphtuuMHveVBQkAYNGqTvfe97evjhh3tiXAAAAL1Kt0rT8ePHe3ocAAAAvVqPXtMEAABwrurWTNO9995rO7t48WJbuZKSEv385z/XjBkzrB8DNk1T999/v5544gl5PB6lpqbqN7/5ja644grrdT6fT0VFRXr22WfV2tqq0aNH67HHHtNFF11kZTwej/Lz861rsbKzs/Xoo4/qggsusDL19fW6++679corrygsLEw5OTlatGiRQkNDbR8rAAA4d3WrNL3++uvavn27Pv30UyUmJkqS3nnnHQUHB+vqq6+2ckFBQba2t23bNj3xxBO68sor/ZYvWLBAixcv1sqVK/XNb35TDz74oMaOHavdu3crIiJCklRQUKA//elPKisr08CBA1VYWKisrCzV1NQoODhYkpSTk6N9+/apvLxckjR16lS53W796U9/kiR1dHQoMzNTgwYNUlVVlT766CNNmjRJpmnq0Ucf7c5bBAAAzjHdKk3jx49XRESEnnrqKUVFRUn6bDbnRz/6kf7zP/9ThYWFtrd19OhR3XbbbVq+fLkefPBBa7lpmnrkkUd033336aabbpIkPfXUU4qNjdUzzzyjadOmyev16sknn9SqVas0ZswYSdLq1asVFxenl156SRkZGdq1a5fKy8tVXV2t1NRUSdLy5cuVlpam3bt3KzExURUVFXrrrbfU0NAgl8slSXr44Yc1efJkPfTQQ4qMjOzO2wQAAM4h3bqm6eGHH1ZJSYlVmCQpKipKDz744Gl/e+7uu+9WZmamVXpO2LNnj5qampSenm4tczgcGjlypDZt2iRJqqmpUXt7u1/G5XIpOTnZymzevFmGYViFSZKGDRsmwzD8MsnJyVZhkqSMjAz5fD7V1NR84dh9Pp9aWlr8HgAA4NzUrdLU0tKiAwcOdFne3NysI0eO2N5OWVmZtm/frpKSki7rmpqaJEmxsbF+y2NjY611TU1NCg0N9StvJ8vExMR02X5MTIxfpvN+oqKiFBoaamVOpqSkRIZhWI+4uLgvO2QAANBHdas03XjjjfrRj36kP/zhD9q3b5/27dunP/zhD5oyZYr1UdqXaWho0IwZM7R69Wqdf/75X5jrfF2UaZpfeq1U58zJ8t3JdDZnzhx5vV7r0dDQcMpxAQCAvqtb1zQ9/vjjKioq0u2336729vbPNhQSoilTpmjhwoW2tlFTU6Pm5malpKRYyzo6OvS3v/1NpaWl2r17t6TPZoEuvPBCK9Pc3GzNCjmdTrW1tcnj8fjNNjU3N2v48OFW5mSzYgcPHvTbzpYtW/zWezwetbe3d5mB+jyHwyGHw2HreAEAQN/WrZmm/v3767HHHtNHH31kfZPu0KFDeuyxxxQeHm5rG6NHj9aOHTtUW1trPYYOHarbbrtNtbW1uuyyy+R0OlVZWWm9pq2tTRs3brQKUUpKivr16+eXaWxsVF1dnZVJS0uT1+vV1q1brcyWLVvk9Xr9MnV1dWpsbLQyFRUVcjgcfqUOAAB8dXVrpumExsZGNTY26tprr1VYWJitj85OiIiIUHJyst+y8PBwDRw40FpeUFCg4uJiJSQkKCEhQcXFxerfv79ycnIkSYZhaMqUKSosLNTAgQM1YMAAFRUVaciQIdaF5YMHD9a4ceOUm5urZcuWSfrslgNZWVnW7RLS09OVlJQkt9uthQsX6tChQyoqKlJubi7fnAMAAJK6WZo++ugjTZgwQa+++qqCgoL07rvv6rLLLtOPf/xjXXDBBT32+3MzZ85Ua2ur8vLyrJtbVlRUWPdokqQlS5YoJCREEyZMsG5uuXLlSuseTZK0Zs0a5efnW9+yy87OVmlpqbU+ODhY69evV15enkaMGOF3c0sAAABJCjJN0zzdF91xxx1qbm7Wb3/7Ww0ePFhvvPGGLrvsMlVUVOinP/2pdu7ceSbG2uu1tLTIMAx5vd4en6G6dPb6Ht0eTu79eZmBHgIA4Cyz+/e7WzNNFRUV+stf/uL3UyWSlJCQoL1793ZnkwAAAL1aty4EP3bsmPr3799l+Ycffsi3yQAAwDmpW6Xp2muv1dNPP209DwoK0vHjx7Vw4UJdd911PTY4AACA3qJbH88tXLhQo0aN0muvvaa2tjbNnDlTO3fu1KFDh/SPf/yjp8cIAAAQcN2aaUpKStKbb76p7373uxo7dqyOHTumm266Sa+//rr+4z/+o6fHCAAAEHCnPdN04gdyly1bpvvvv/9MjAkAAKDXOe2Zpn79+qmurs72TSwBAADOBd36eO6OO+7Qk08+2dNjAQAA6LW6dSF4W1ubfvvb36qyslJDhw7t8ntzixcv7pHBAQAA9BanVZree+89XXrppaqrq9PVV18tSXrnnXf8MnxsBwAAzkWnVZoSEhLU2NioV199VZI0ceJE/frXv1ZsbOwZGRwAAEBvcVrXNHX+mboNGzbo2LFjPTogAACA3qhbF4Kf0I3f+gUAAOiTTqs0BQUFdblmiWuYAADAV8FpXdNkmqYmT55s/SjvJ598orvuuqvLt+eef/75nhshAABAL3BapWnSpEl+z2+//fYeHQwAAEBvdVqlacWKFWdqHAAAAL3av3UhOAAAwFcFpQkAAMAGShMAAIANlCYAAAAbKE0AAAA2UJoAAABsoDQBAADYQGkCAACwgdIEAABgA6UJAADABkoTAACADZQmAAAAGyhNAAAANlCaAAAAbAhoaVq6dKmuvPJKRUZGKjIyUmlpadqwYYO13jRNzZ07Vy6XS2FhYRo1apR27tzptw2fz6fp06crOjpa4eHhys7O1r59+/wyHo9HbrdbhmHIMAy53W4dPnzYL1NfX6/x48crPDxc0dHRys/PV1tb2xk7dgAA0LcEtDRddNFFmjdvnl577TW99tpr+t73vqcf/OAHVjFasGCBFi9erNLSUm3btk1Op1Njx47VkSNHrG0UFBRo3bp1KisrU1VVlY4ePaqsrCx1dHRYmZycHNXW1qq8vFzl5eWqra2V2+221nd0dCgzM1PHjh1TVVWVysrKtHbtWhUWFp69NwMAAPRqQaZpmoEexOcNGDBACxcu1J133imXy6WCggLNmjVL0mezSrGxsZo/f76mTZsmr9erQYMGadWqVZo4caIkaf/+/YqLi9OLL76ojIwM7dq1S0lJSaqurlZqaqokqbq6WmlpaXr77beVmJioDRs2KCsrSw0NDXK5XJKksrIyTZ48Wc3NzYqMjLQ19paWFhmGIa/Xa/s1dl06e32Pbg8n9/68zEAPAQBwltn9+91rrmnq6OhQWVmZjh07prS0NO3Zs0dNTU1KT0+3Mg6HQyNHjtSmTZskSTU1NWpvb/fLuFwuJScnW5nNmzfLMAyrMEnSsGHDZBiGXyY5OdkqTJKUkZEhn8+nmpqaLxyzz+dTS0uL3wMAAJybAl6aduzYoa997WtyOBy66667tG7dOiUlJampqUmSFBsb65ePjY211jU1NSk0NFRRUVGnzMTExHTZb0xMjF+m836ioqIUGhpqZU6mpKTEuk7KMAzFxcWd5tEDAIC+IuClKTExUbW1taqurtZPfvITTZo0SW+99Za1PigoyC9vmmaXZZ11zpws351MZ3PmzJHX67UeDQ0NpxwXAADouwJemkJDQ/WNb3xDQ4cOVUlJib71rW/pV7/6lZxOpyR1melpbm62ZoWcTqfa2trk8XhOmTlw4ECX/R48eNAv03k/Ho9H7e3tXWagPs/hcFjf/DvxAAAA56aAl6bOTNOUz+dTfHy8nE6nKisrrXVtbW3auHGjhg8fLklKSUlRv379/DKNjY2qq6uzMmlpafJ6vdq6dauV2bJli7xer1+mrq5OjY2NVqaiokIOh0MpKSln9HgBAEDfEBLInf/85z/X9ddfr7i4OB05ckRlZWX661//qvLycgUFBamgoEDFxcVKSEhQQkKCiouL1b9/f+Xk5EiSDMPQlClTVFhYqIEDB2rAgAEqKirSkCFDNGbMGEnS4MGDNW7cOOXm5mrZsmWSpKlTpyorK0uJiYmSpPT0dCUlJcntdmvhwoU6dOiQioqKlJuby+wRAACQFODSdODAAbndbjU2NsowDF155ZUqLy/X2LFjJUkzZ85Ua2ur8vLy5PF4lJqaqoqKCkVERFjbWLJkiUJCQjRhwgS1trZq9OjRWrlypYKDg63MmjVrlJ+fb33LLjs7W6Wlpdb64OBgrV+/Xnl5eRoxYoTCwsKUk5OjRYsWnaV3AgAA9Ha97j5NfRn3aer7uE8TAHz19Ln7NAEAAPRmlCYAAAAbKE0AAAA2UJoAAABsoDQBAADYQGkCAACwgdIEAABgA6UJAADABkoTAACADZQmAAAAGyhNAAAANlCaAAAAbKA0AQAA2EBpAgAAsIHSBAAAYAOlCQAAwAZKEwAAgA2UJgAAABsoTQAAADZQmgAAAGygNAEAANhAaQIAALCB0gQAAGADpQkAAMAGShMAAIANlCYAAAAbKE0AAAA2UJoAAABsoDQBAADYQGkCAACwgdIEAABgA6UJAADAhoCWppKSEn3nO99RRESEYmJidMMNN2j37t1+GdM0NXfuXLlcLoWFhWnUqFHauXOnX8bn82n69OmKjo5WeHi4srOztW/fPr+Mx+OR2+2WYRgyDENut1uHDx/2y9TX12v8+PEKDw9XdHS08vPz1dbWdkaOHQAA9C0BLU0bN27U3XffrerqalVWVurTTz9Venq6jh07ZmUWLFigxYsXq7S0VNu2bZPT6dTYsWN15MgRK1NQUKB169aprKxMVVVVOnr0qLKystTR0WFlcnJyVFtbq/LycpWXl6u2tlZut9ta39HRoczMTB07dkxVVVUqKyvT2rVrVVhYeHbeDAAA0KsFmaZpBnoQJxw8eFAxMTHauHGjrr32WpmmKZfLpYKCAs2aNUvSZ7NKsbGxmj9/vqZNmyav16tBgwZp1apVmjhxoiRp//79iouL04svvqiMjAzt2rVLSUlJqq6uVmpqqiSpurpaaWlpevvtt5WYmKgNGzYoKytLDQ0NcrlckqSysjJNnjxZzc3NioyM/NLxt7S0yDAMeb1eW/nTcens9T26PZzc+/MyAz0EAMBZZvfvd6+6psnr9UqSBgwYIEnas2ePmpqalJ6ebmUcDodGjhypTZs2SZJqamrU3t7ul3G5XEpOTrYymzdvlmEYVmGSpGHDhskwDL9McnKyVZgkKSMjQz6fTzU1NScdr8/nU0tLi98DAACcm3pNaTJNU/fee6+uueYaJScnS5KampokSbGxsX7Z2NhYa11TU5NCQ0MVFRV1ykxMTEyXfcbExPhlOu8nKipKoaGhVqazkpIS6xopwzAUFxd3uocNAAD6iF5Tmu655x69+eabevbZZ7usCwoK8ntummaXZZ11zpws353M582ZM0der9d6NDQ0nHJMAACg7+oVpWn69Ol64YUX9Oqrr+qiiy6yljudTknqMtPT3NxszQo5nU61tbXJ4/GcMnPgwIEu+z148KBfpvN+PB6P2tvbu8xAneBwOBQZGen3AAAA56aAlibTNHXPPffo+eef1yuvvKL4+Hi/9fHx8XI6naqsrLSWtbW1aePGjRo+fLgkKSUlRf369fPLNDY2qq6uzsqkpaXJ6/Vq69atVmbLli3yer1+mbq6OjU2NlqZiooKORwOpaSk9PzBAwCAPiUkkDu/++679cwzz+h//ud/FBERYc30GIahsLAwBQUFqaCgQMXFxUpISFBCQoKKi4vVv39/5eTkWNkpU6aosLBQAwcO1IABA1RUVKQhQ4ZozJgxkqTBgwdr3Lhxys3N1bJlyyRJU6dOVVZWlhITEyVJ6enpSkpKktvt1sKFC3Xo0CEVFRUpNzeXGSQAABDY0rR06VJJ0qhRo/yWr1ixQpMnT5YkzZw5U62trcrLy5PH41FqaqoqKioUERFh5ZcsWaKQkBBNmDBBra2tGj16tFauXKng4GArs2bNGuXn51vfssvOzlZpaam1Pjg4WOvXr1deXp5GjBihsLAw5eTkaNGiRWfo6AEAQF/Sq+7T1Ndxn6a+j/s0AcBXT5+8TxMAAEBvRWkCAACwgdIEAABgA6UJAADABkoTAACADZQmAAAAGyhNAAAANlCaAAAAbKA0AQAA2EBpAgAAsIHSBAAAYAOlCQAAwAZKEwAAgA2UJgAAABsoTQAAADZQmgAAAGygNAEAANhAaQIAALCB0gQAAGADpQkAAMAGShMAAIANlCYAAAAbKE0AAAA2UJoAAABsoDQBAADYQGkCAACwgdIEAABgA6UJAADABkoTAACADZQmAAAAGyhNAAAANgS0NP3tb3/T+PHj5XK5FBQUpD/+8Y9+603T1Ny5c+VyuRQWFqZRo0Zp586dfhmfz6fp06crOjpa4eHhys7O1r59+/wyHo9HbrdbhmHIMAy53W4dPnzYL1NfX6/x48crPDxc0dHRys/PV1tb25k4bAAA0AcFtDQdO3ZM3/rWt1RaWnrS9QsWLNDixYtVWlqqbdu2yel0auzYsTpy5IiVKSgo0Lp161RWVqaqqiodPXpUWVlZ6ujosDI5OTmqra1VeXm5ysvLVVtbK7fbba3v6OhQZmamjh07pqqqKpWVlWnt2rUqLCw8cwcPAAD6lCDTNM1AD0KSgoKCtG7dOt1www2SPptlcrlcKigo0KxZsyR9NqsUGxur+fPna9q0afJ6vRo0aJBWrVqliRMnSpL279+vuLg4vfjii8rIyNCuXbuUlJSk6upqpaamSpKqq6uVlpamt99+W4mJidqwYYOysrLU0NAgl8slSSorK9PkyZPV3NysyMhIW8fQ0tIiwzDk9Xptv8auS2ev79Ht4eTen5cZ6CEAAM4yu3+/e+01TXv27FFTU5PS09OtZQ6HQyNHjtSmTZskSTU1NWpvb/fLuFwuJScnW5nNmzfLMAyrMEnSsGHDZBiGXyY5OdkqTJKUkZEhn8+nmpqaM3qcAACgbwgJ9AC+SFNTkyQpNjbWb3lsbKz27t1rZUJDQxUVFdUlc+L1TU1NiomJ6bL9mJgYv0zn/URFRSk0NNTKnIzP55PP57Oet7S02D08AADQx/TamaYTgoKC/J6bptllWWedMyfLdyfTWUlJiXVxuWEYiouLO+W4AABA39VrS5PT6ZSkLjM9zc3N1qyQ0+lUW1ubPB7PKTMHDhzosv2DBw/6ZTrvx+PxqL29vcsM1OfNmTNHXq/XejQ0NJzmUQIAgL6i15am+Ph4OZ1OVVZWWsva2tq0ceNGDR8+XJKUkpKifv36+WUaGxtVV1dnZdLS0uT1erV161Yrs2XLFnm9Xr9MXV2dGhsbrUxFRYUcDodSUlK+cIwOh0ORkZF+DwAAcG4K6DVNR48e1T//+U/r+Z49e1RbW6sBAwbo4osvVkFBgYqLi5WQkKCEhAQVFxerf//+ysnJkSQZhqEpU6aosLBQAwcO1IABA1RUVKQhQ4ZozJgxkqTBgwdr3Lhxys3N1bJlyyRJU6dOVVZWlhITEyVJ6enpSkpKktvt1sKFC3Xo0CEVFRUpNzeXIgQAACQFuDS99tpruu6666zn9957ryRp0qRJWrlypWbOnKnW1lbl5eXJ4/EoNTVVFRUVioiIsF6zZMkShYSEaMKECWptbdXo0aO1cuVKBQcHW5k1a9YoPz/f+pZddna2372hgoODtX79euXl5WnEiBEKCwtTTk6OFi1adKbfAgAA0Ef0mvs0nQu4TxNgH/fEAtBb9Pn7NAEAAPQmlCYAAAAbKE0AAAA2UJoAAABsoDQBAADYQGkCAACwgdIEAABgA6UJAADABkoTAACADZQmAAAAGyhNAAAANlCaAAAAbKA0AQAA2EBpAgAAsIHSBAAAYAOlCQAAwAZKEwAAgA2UJgAAABsoTQAAADZQmgAAAGygNAEAANgQEugBAPhqunT2+kAP4Svh/XmZgR4CcM5gpgkAAMAGShMAAIANlCYAAAAbKE0AAAA2UJoAAABsoDQBAADYQGkCAACwgdIEAABgA6UJAADABkoTAACADZSmTh577DHFx8fr/PPPV0pKiv7+978HekgAAKAXoDR9znPPPaeCggLdd999ev311/Wf//mfuv7661VfXx/ooQEAgAALMk3TDPQgeovU1FRdffXVWrp0qbVs8ODBuuGGG1RSUvKlr29paZFhGPJ6vYqMjOzRsfHjpgDQe/HDyH2b3b/fIWdxTL1aW1ubampqNHv2bL/l6enp2rRp00lf4/P55PP5rOder1fSZ29+Tzvu+7jHtwkA6Bln4r/7OHtOnL8vm0eiNP2vDz/8UB0dHYqNjfVbHhsbq6amppO+pqSkRPfff3+X5XFxcWdkjACA3sl4JNAjQE84cuSIDMP4wvWUpk6CgoL8npum2WXZCXPmzNG9995rPT9+/LgOHTqkgQMHfuFruqOlpUVxcXFqaGjo8Y/9cHo4F70H56L34Fz0HpyL7jFNU0eOHJHL5TpljtL0v6KjoxUcHNxlVqm5ubnL7NMJDodDDofDb9kFF1xwpoaoyMhI/hH0EpyL3oNz0XtwLnoPzsXpO9UM0wl8e+5/hYaGKiUlRZWVlX7LKysrNXz48ACNCgAA9BbMNH3OvffeK7fbraFDhyotLU1PPPGE6uvrdddddwV6aAAAIMAoTZ8zceJEffTRR3rggQfU2Nio5ORkvfjii7rkkksCOi6Hw6Ff/vKXXT4KxNnHueg9OBe9B+ei9+BcnFncpwkAAMAGrmkCAACwgdIEAABgA6UJAADABkoTAACADZSmPuCxxx5TfHy8zj//fKWkpOjvf/97oIfUZ5WUlOg73/mOIiIiFBMToxtuuEG7d+/2y5imqblz58rlciksLEyjRo3Szp07/TI+n0/Tp09XdHS0wsPDlZ2drX379vllPB6P3G63DMOQYRhyu906fPjwmT7EPqukpERBQUEqKCiwlnEuzp4PPvhAt99+uwYOHKj+/fvr29/+tmpqaqz1nIuz49NPP9UvfvELxcfHKywsTJdddpkeeOABHT9+3MpwLgLIRK9WVlZm9uvXz1y+fLn51ltvmTNmzDDDw8PNvXv3BnpofVJGRoa5YsUKs66uzqytrTUzMzPNiy++2Dx69KiVmTdvnhkREWGuXbvW3LFjhzlx4kTzwgsvNFtaWqzMXXfdZX796183Kysrze3bt5vXXXed+a1vfcv89NNPrcy4cePM5ORkc9OmTeamTZvM5ORkMysr66web1+xdetW89JLLzWvvPJKc8aMGdZyzsXZcejQIfOSSy4xJ0+ebG7ZssXcs2eP+dJLL5n//Oc/rQzn4ux48MEHzYEDB5p//vOfzT179pi///3vza997WvmI488YmU4F4FDaerlvvvd75p33XWX37LLL7/cnD17doBGdG5pbm42JZkbN240TdM0jx8/bjqdTnPevHlW5pNPPjENwzAff/xx0zRN8/Dhw2a/fv3MsrIyK/PBBx+Y5513nlleXm6apmm+9dZbpiSzurraymzevNmUZL799ttn49D6jCNHjpgJCQlmZWWlOXLkSKs0cS7OnlmzZpnXXHPNF67nXJw9mZmZ5p133um37KabbjJvv/120zQ5F4HGx3O9WFtbm2pqapSenu63PD09XZs2bQrQqM4tXq9XkjRgwABJ0p49e9TU1OT3njscDo0cOdJ6z2tqatTe3u6XcblcSk5OtjKbN2+WYRhKTU21MsOGDZNhGJy7Tu6++25lZmZqzJgxfss5F2fPCy+8oKFDh+rmm29WTEyMrrrqKi1fvtxaz7k4e6655hq9/PLLeueddyRJb7zxhqqqqvT9739fEuci0LgjeC/24YcfqqOjo8sPBsfGxnb5YWGcPtM0de+99+qaa65RcnKyJFnv68ne871791qZ0NBQRUVFdcmceH1TU5NiYmK67DMmJoZz9zllZWXavn27tm3b1mUd5+Lsee+997R06VLde++9+vnPf66tW7cqPz9fDodDd9xxB+fiLJo1a5a8Xq8uv/xyBQcHq6OjQw899JBuvfVWSfy7CDRKUx8QFBTk99w0zS7LcPruuecevfnmm6qqquqyrjvveefMyfKcu//T0NCgGTNmqKKiQueff/4X5jgXZ97x48c1dOhQFRcXS5Kuuuoq7dy5U0uXLtUdd9xh5TgXZ95zzz2n1atX65lnntEVV1yh2tpaFRQUyOVyadKkSVaOcxEYfDzXi0VHRys4OLhL629ubu7yfxk4PdOnT9cLL7ygV199VRdddJG13Ol0StIp33On06m2tjZ5PJ5TZg4cONBlvwcPHuTc/a+amho1NzcrJSVFISEhCgkJ0caNG/XrX/9aISEh1vvEuTjzLrzwQiUlJfktGzx4sOrr6yXx7+Js+tnPfqbZs2frlltu0ZAhQ+R2u/XTn/5UJSUlkjgXgUZp6sVCQ0OVkpKiyspKv+WVlZUaPnx4gEbVt5mmqXvuuUfPP/+8XnnlFcXHx/utj4+Pl9Pp9HvP29ratHHjRus9T0lJUb9+/fwyjY2NqqurszJpaWnyer3aunWrldmyZYu8Xi/n7n+NHj1aO3bsUG1trfUYOnSobrvtNtXW1uqyyy7jXJwlI0aM6HLrjXfeecf6sXL+XZw9H3/8sc47z/9Pc3BwsHXLAc5FgAXg4nOchhO3HHjyySfNt956yywoKDDDw8PN999/P9BD65N+8pOfmIZhmH/961/NxsZG6/Hxxx9bmXnz5pmGYZjPP/+8uWPHDvPWW2896dd5L7roIvOll14yt2/fbn7ve9876dd5r7zySnPz5s3m5s2bzSFDhvB13i/x+W/PmSbn4mzZunWrGRISYj700EPmu+++a65Zs8bs37+/uXr1aivDuTg7Jk2aZH7961+3bjnw/PPPm9HR0ebMmTOtDOcicChNfcBvfvMb85JLLjFDQ0PNq6++2vp6PE6fpJM+VqxYYWWOHz9u/vKXvzSdTqfpcDjMa6+91tyxY4ffdlpbW8177rnHHDBggBkWFmZmZWWZ9fX1fpmPPvrIvO2228yIiAgzIiLCvO2220yPx3MWjrLv6lyaOBdnz5/+9CczOTnZdDgc5uWXX24+8cQTfus5F2dHS0uLOWPGDPPiiy82zz//fPOyyy4z77vvPtPn81kZzkXgBJmmaQZypgsAAKAv4JomAAAAGyhNAAAANlCaAAAAbKA0AQAA2EBpAgAAsIHSBAAAYAOlCQAAwAZKEwAAgA2UJgAAABsoTQAAADZQmgAAAGygNAEAANjw/wEJ3KOSTYesIQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "for c in numerical_features:\n",
    "    print(c)\n",
    "    df[c].plot.hist(bins=5)\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If for some histograms the values are heavily placed in the first bin, it is good to check for outliers, either checking the min-max values of those particular features and/or explore value ranges."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Age upon Intake Days\n",
      "min: 0 max: 9125\n",
      "Age upon Outcome Days\n",
      "min: 0 max: 9125\n"
     ]
    }
   ],
   "source": [
    "for c in numerical_features:\n",
    "    print(c)\n",
    "    print('min:', df[c].min(), 'max:', df[c].max())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With __value_counts()__ function, we can increase the number of histogram bins to 10 for more bins for a more refined view of the numerical features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Age upon Intake Days\n",
      "(-9.126, 912.5]     74835\n",
      "(912.5, 1825.0]     10647\n",
      "(1825.0, 2737.5]     3471\n",
      "(2737.5, 3650.0]     3998\n",
      "(3650.0, 4562.5]     1234\n",
      "(4562.5, 5475.0]     1031\n",
      "(5475.0, 6387.5]      183\n",
      "(6387.5, 7300.0]       79\n",
      "(7300.0, 8212.5]        5\n",
      "(8212.5, 9125.0]        2\n",
      "Name: count, dtype: int64\n",
      "Age upon Outcome Days\n",
      "(-9.126, 912.5]     74642\n",
      "(912.5, 1825.0]     10699\n",
      "(1825.0, 2737.5]     3465\n",
      "(2737.5, 3650.0]     4080\n",
      "(3650.0, 4562.5]     1263\n",
      "(4562.5, 5475.0]     1061\n",
      "(5475.0, 6387.5]      187\n",
      "(6387.5, 7300.0]       81\n",
      "(7300.0, 8212.5]        5\n",
      "(8212.5, 9125.0]        2\n",
      "Name: count, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "for c in numerical_features: \n",
    "    print(c)\n",
    "    print(df[c].value_counts(bins=10, sort=False))\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If any outliers are identified as very likely wrong values, dropping them could improve the numerical values histograms, and later overall model performance. While a good rule of thumb is that anything not in the range of (Q1 - 1.5 IQR) and (Q3 + 1.5 IQR) is an outlier, other rules for removing 'outliers' should be considered as well. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's check missing values for these numerical features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Age upon Intake Days     0\n",
      "Age upon Outcome Days    0\n",
      "dtype: int64\n"
     ]
    }
   ],
   "source": [
    "print(df[numerical_features].isna().sum())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If any missing values, as a quick fix, we can apply mean imputation. This will replace the missing values with the mean value of the corresponding column.\n",
    "\n",
    "__Note__: The statistically correct way to perform mean/mode imputation before training an ML model is to compute the column-wise means on the training data only, and then use these values to impute missing data in both the train and test sets. So, you'll need to split your dataset first. Same goes for any other transformations we would like to apply to these numerical features, such as scaling. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Cleaning categorical features \n",
    "\n",
    "Let's also examine the categorical features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sex upon Outcome\n",
      "['Neutered Male' 'Intact Male' 'Intact Female' 'Unknown' 'Spayed Female'\n",
      " nan]\n",
      "Intake Type\n",
      "['Owner Surrender' 'Stray' 'Wildlife' 'Public Assist' 'Euthanasia Request'\n",
      " 'Abandoned']\n",
      "Intake Condition\n",
      "['Normal' 'Nursing' 'Sick' 'Injured' 'Aged' 'Feral' 'Pregnant' 'Other'\n",
      " 'Behavior' 'Medical']\n",
      "Pet Type\n",
      "['Cat' 'Dog' 'Other' 'Bird' 'Livestock']\n",
      "Sex upon Intake\n",
      "['Neutered Male' 'Intact Male' 'Intact Female' 'Unknown' 'Spayed Female'\n",
      " nan]\n"
     ]
    }
   ],
   "source": [
    "for c in categorical_features:\n",
    "    print(c)\n",
    "    print(df[c].unique()) #value_counts())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Note on boolean type features__: Some categories might be of boolean type, like __False__ and __True__. The booleans will raise errors when trying to encode the categoricals with sklearn encoders, none of which accept boolean types. If using pandas get_dummies to one-hot encode the categoricals, there's no need to convert the booleans. However, get_dummies is trickier to use with sklearn's Pipeline and GridSearch. \n",
    "\n",
    "One way to deal with the booleans is to convert them to strings, by using a mask and a map changing only the booleans. Another way to handle the booleans is to convert them to strings by changing the type of all categoricals to 'str'. This will also affect the nans, basically performing imputation of the nans with a 'nans' placeholder value! \n",
    "\n",
    "Applying the type conversion to both categoricals and text features, takes care of the nans in the text fields as well. In case other imputations are planned for the categoricals and/or test fields, notice that the masking shown above leaves the nans unchanged."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "df[categorical_features + text_features] = df[categorical_features + text_features].astype('str')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's have a check on missing values for the categorical features (and text features here)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sex upon Outcome    0\n",
      "Intake Type         0\n",
      "Intake Condition    0\n",
      "Pet Type            0\n",
      "Sex upon Intake     0\n",
      "Name                0\n",
      "Found Location      0\n",
      "Breed               0\n",
      "Color               0\n",
      "dtype: int64\n"
     ]
    }
   ],
   "source": [
    "print(df[categorical_features + text_features].isna().sum())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Converting categoricals into useful numerical features will also have to wait until after the train/test split."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Cleaning text features \n",
    "\n",
    "Also a good idea to look at the text fields. Text cleaning can be performed here, before train/test split, with less code. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Name\n",
      "['Chunk' 'Gizmo' 'nan' ... '*Lingonberry' 'Guawp' '*Squanchy']\n",
      "Found Location\n",
      "['Austin (TX)' '7201 Levander Loop in Austin (TX)'\n",
      " '12034 Research in Austin (TX)' ... '4612 Sherwyn Drive in Austin (TX)'\n",
      " '16010 Voelker Ln in Austin (TX)' '2211 Santa Rita Street in Austin (TX)']\n",
      "Breed\n",
      "['Domestic Shorthair Mix' 'Chihuahua Shorthair Mix' 'Domestic Shorthair'\n",
      " ... 'Unknown' 'Bichon Frise/Lhasa Apso' 'Treeing Cur']\n",
      "Color\n",
      "['Brown Tabby/White' 'White/Brown' 'Orange Tabby' 'Black'\n",
      " 'White/Orange Tabby' 'Blue/White' 'Brown Tabby' 'Gray' 'Calico'\n",
      " 'Brown/Black' 'White/Tan' 'White' 'Brown' 'Black/White' 'Brown/White'\n",
      " 'Black/Brown' 'Chocolate/White' 'Red' 'White/White' 'Brown Brindle/White'\n",
      " 'Gray/Black' 'Tortie' 'Tan' 'White/Blue Tabby' 'Brown/Brown' 'Black/Gray'\n",
      " 'Blue' 'Cream Tabby' 'Brown/Gray' 'Blue Tabby/White' 'Red/White'\n",
      " 'Orange Tabby/White' 'Brown Merle/White' 'Tricolor' 'Apricot' 'Black/Tan'\n",
      " 'Tortie Point' 'Tan/Black' 'Torbie/Brown Tabby' 'White/Black'\n",
      " 'Blue Tabby' 'Blue Tick' 'White/Gray' 'Black/Tricolor' 'Chocolate/Tan'\n",
      " 'White/Brown Tabby' 'White/Brown Brindle' 'Lynx Point' 'Buff' 'Torbie'\n",
      " 'White/Buff' 'Brown Brindle' 'Cream' 'White/Blue' 'Blue/Tan'\n",
      " 'Black Brindle/White' 'Black/Yellow Brindle' 'Chocolate/Black'\n",
      " 'Black/Red' 'Fawn/White' 'Chocolate' 'Blue/Brown Brindle' 'Tan/White'\n",
      " 'Cream Tabby/White' 'Tan/Gray' 'Sable' 'Red/Buff' 'Blue Merle'\n",
      " 'Lynx Point/White' 'Yellow' 'Black/Brown Brindle' 'Brown/Tan'\n",
      " 'Silver Tabby' 'White/Red' 'Brown/Orange' 'Tricolor/White' 'Sable/Black'\n",
      " 'Gray/White' 'Orange/White' 'Brown Tiger/Brown' 'Brown Tabby/Black'\n",
      " 'Torbie/White' 'Yellow Brindle' 'Cream/White' 'Brown Brindle/Black'\n",
      " 'Black/Chocolate' 'Gold/White' 'White/Orange' 'Black Tabby'\n",
      " 'Tricolor/Brown' 'Seal Point/Gray' 'White/Tricolor' 'Silver/Tan'\n",
      " 'Gray Tabby/White' 'Black Brindle' 'Brown/Tricolor' 'Black Tabby/White'\n",
      " 'Yellow/White' 'Cream/Black' 'Gray/Tortie' 'Flame Point' 'Sable/White'\n",
      " 'Seal Point' 'Chocolate Point' 'Red/Tan' 'Gray/Tan' 'Calico Point/Gray'\n",
      " 'Black/Black' 'Red/Cream' 'White/Red Merle' 'Tortie/White' 'Red/Black'\n",
      " 'Green/Silver' 'Brown/Red' 'Gray Tabby' 'Green/Gray' 'Gray/Brown'\n",
      " 'Gray/Blue Merle' 'White/Blue Merle' 'Blue Cream' 'Silver Tabby/White'\n",
      " 'Black/Cream' 'Lilac Point' 'Gray Tabby/Black' 'Brown Merle' 'Gold/Cream'\n",
      " 'Gold' 'Blue Merle/Tricolor' 'Buff/White' 'White/Cream' 'Red Merle/White'\n",
      " 'Fawn/Black' 'Yellow/Yellow' 'Sable/Brown' 'Black/Black Tabby'\n",
      " 'White/Gray Tabby' 'Calico/White' 'Tricolor/Black' 'Fawn' 'Blue/Blue'\n",
      " 'White/Chocolate' 'Tan/Fawn' 'Blue Merle/White' 'Lynx Point/Blue'\n",
      " 'Blue Merle/Brown' 'Blue Merle/Tan' 'White/Seal Point' 'Liver/Tan'\n",
      " 'Blue Point/White' 'Liver/White' 'Chocolate Point/White' 'Gray/Pink'\n",
      " 'Black Brindle/Blue' 'Black Smoke' 'Brown Brindle/Red Tick' 'Cream/Brown'\n",
      " 'Black/Blue Tick' 'Red Tick/Blue Tick' 'White/Yellow' 'Orange'\n",
      " 'Torbie/Brown' 'Sable/Tan' 'Yellow Brindle/White' 'Gray/Orange'\n",
      " 'Calico Point' 'Red Tick' 'White/Red Tick' 'Blue Tick/Tan' 'Brown/Cream'\n",
      " 'Cream/Tan' 'Tan/Brown' 'Buff/Brown' 'Tan/Tan' 'Chocolate/Tricolor'\n",
      " 'Calico Point/White' 'Brown Brindle/Brown Brindle' 'Green/Brown'\n",
      " 'Black/Silver' 'White/Calico' 'Brown/Chocolate' 'Cream/Silver'\n",
      " 'White/Brown Merle' 'Red/Brown' 'White/Fawn' 'White/Gray Tiger'\n",
      " 'Tan/Gold' 'Tan/Red' 'Tan/Silver' 'Lilac Point/White' 'Buff/Black'\n",
      " 'Cream/Brown Merle' 'White/Black Brindle' 'Silver' 'Lilac Point/Gray'\n",
      " 'Black Smoke/White' 'Pink' 'Blue Tick/Tricolor' 'Blue Tick/Black'\n",
      " 'Seal Point/White' 'Blue Point' 'Silver/Brown' 'Fawn/Brown'\n",
      " 'Black Brindle/Brown' 'Blue Merle/Black' 'Blue Cream/White'\n",
      " 'White/Blue Cream' 'Gray/Gray' 'Tortie/Tortie' 'Green' 'Brown/Buff'\n",
      " 'Chocolate/Brown Tabby' 'Tortie/Blue Cream' 'Brown/Fawn' 'White/Tortie'\n",
      " 'Orange Tabby/Orange Tabby' 'Tortie/Black' 'White/Cream Tabby'\n",
      " 'Tan/Cream' 'Red/Yellow' 'Blue/Tortie' 'Lynx Point/Brown Tabby'\n",
      " 'Black Tabby/Orange' 'Blue/Tricolor' 'Black/Blue' 'White/Agouti'\n",
      " 'Gold/Yellow' 'Chocolate/Fawn' 'Orange/Orange Tabby'\n",
      " 'Tricolor/Blue Merle' 'Brown/Brown Tabby' 'Black/Orange'\n",
      " 'Cream/Blue Point' 'Calico/Tricolor' 'Agouti' 'Calico/Black'\n",
      " 'Brown Brindle/Brown' 'Lilac Point/Black' 'Tan/Blue Merle'\n",
      " 'Blue Tabby/Black' 'Silver/Black' 'Tan/Blue' 'Black/Yellow'\n",
      " 'Yellow/Green' 'Flame Point/Cream' 'Agouti/White' 'Brown/Green'\n",
      " 'Yellow/Black' 'Torbie/Blue Tabby' 'White/Black Tabby'\n",
      " 'Blue Merle/Red Merle' 'Cream/Orange' 'Gray/Cream' 'Silver/Chocolate'\n",
      " 'Tan/Tricolor' 'Red Merle' 'Chocolate/Cream' 'Tan/Buff'\n",
      " 'Brown Tiger/White' 'Blue/Gray' 'Gray Tabby/Gray' 'Chocolate/Red'\n",
      " 'Black Brindle/Black' 'Gray/Blue' 'Tricolor/Blue' 'Red Tick/Brown'\n",
      " 'Yellow/Blue' 'Gray/Yellow' 'Brown/Liver' 'Red/Red' 'Silver/Orange'\n",
      " 'Black/Pink' 'Tricolor/Tan' 'Calico/Brown' 'Tortie Point/Lynx Point'\n",
      " 'Cream/Blue' 'Green/Red' 'Tortie/Orange' 'Gray/Red' 'Black/Black Smoke'\n",
      " 'Buff/Tan' 'Brown/Yellow' 'Fawn/Blue' 'Red/Tricolor' 'Fawn/Cream'\n",
      " 'Silver/Red' 'Brown Tabby/Calico' 'Black/Blue Merle' 'Yellow/Orange'\n",
      " 'Brown Tabby/Brown' 'Blue Tick/Red' 'Seal Point/Cream'\n",
      " 'Cream Tabby/Orange' 'Chocolate/Blue Tick' 'Red/Blue'\n",
      " 'Blue Cream/Blue Tiger' 'Blue/Black' 'Brown Tiger' 'Brown Merle/Tan'\n",
      " 'Fawn/Tan' 'Brown Tabby/Orange' 'Blue Smoke' 'Red Tick/White'\n",
      " 'White/Apricot' 'White/Liver' 'Red/Gold' 'Green/Yellow' 'Red/Red Merle'\n",
      " 'Gold/Tan' 'Fawn/Tricolor' 'Blue Merle/Blue Merle' 'Red/Silver'\n",
      " 'Calico/Orange' 'Brown Tabby/Silver' 'Blue Tick/Red Tick'\n",
      " 'Orange Tabby/Brown' 'Apricot/Brown' 'Red/Red Tick' 'Gray/Tricolor'\n",
      " 'Black/Buff' 'Black/Green' 'Black/Black Brindle' 'Orange/Black'\n",
      " 'Tortie Point/White' 'White/Liver Tick' 'Gold/Brown' 'Red Merle/Black'\n",
      " 'Apricot/White' 'Brown Tabby/Brown Tabby' 'Cream/Seal Point'\n",
      " 'Tan/Red Merle' 'Gray/Green' 'Chocolate/Brown' 'Buff/Cream' 'Buff/Red'\n",
      " 'Brown/Silver' 'Blue Tick/Brown' 'Brown Tabby/Tortie' 'Blue Tiger/White'\n",
      " 'Tan/Chocolate Point' 'Black/Fawn' 'Blue/Brown' 'White/Blue Tick'\n",
      " 'Blue Tick/White' 'Orange Tabby/Orange' 'Blue Tick/Brown Brindle'\n",
      " 'White/Gold' 'Black Smoke/Brown Tabby' 'Red/Gray'\n",
      " 'Brown Brindle/Tricolor' 'Orange/Brown' 'Gray/Gold' 'Liver Tick/White'\n",
      " 'Black Smoke/Black Tiger' 'Brown/Red Merle' 'Sable/Buff'\n",
      " 'Gray Tabby/Brown Tabby' 'Gold/Silver' 'Seal Point/Brown'\n",
      " 'Silver Lynx Point' 'Black/Gold' 'Liver' 'Yellow/Tan' 'Blue Tiger'\n",
      " 'Tan/Yellow' 'Orange/Tan' 'Lynx Point/Tortie Point' 'Brown Tabby/Gray'\n",
      " 'Sable/Cream' 'White/Chocolate Point' 'White/Yellow Brindle'\n",
      " 'Black Tiger/White' 'Calico/Gray Tabby' 'Buff/Gray' 'Tricolor/Silver'\n",
      " 'Cream/Red' 'Gold/Buff' 'Liver Tick' 'Brown Brindle/Tan'\n",
      " 'Tricolor/Blue Tick' 'White/Pink' 'Sable/Gray' 'Brown/Brown Brindle'\n",
      " 'Orange Tabby/Tortie Point' 'Chocolate/Chocolate' 'Yellow/Gray'\n",
      " 'Chocolate Point/Cream' 'Black Brindle/Brown Brindle' 'Yellow/Cream'\n",
      " 'Gold/Black' 'Tan/Yellow Brindle' 'Red Tick/Tricolor'\n",
      " 'Brown Merle/Brown Tabby' 'Flame Point/White' 'Calico/Calico'\n",
      " 'Orange Tabby/Apricot' 'Blue/Calico' 'Brown/Black Smoke' 'Green/Black'\n",
      " 'Calico Point/Lynx Point' 'Torbie/Gray' 'Tortie/Calico'\n",
      " 'Brown Tabby/Black Tabby' 'Tortie/Blue' 'White/Lynx Point'\n",
      " 'Red Tick/Brown Brindle' 'Gold/Gray' 'Silver/White' 'Blue/Cream'\n",
      " 'Blue Tabby/Cream' 'Black Smoke/Blue Tick' 'Tricolor/Cream'\n",
      " 'Gray Tabby/Orange' 'Brown Tabby/Blue' 'Blue Tabby/Buff' 'Tricolor/Red'\n",
      " 'Chocolate/Gold' 'Brown Merle/Brown' 'Cream/Gray' 'Torbie/Calico'\n",
      " 'Yellow/Red' 'Tricolor/Gray' 'White/Silver Tabby' 'Red Tick/Tan'\n",
      " 'Orange/Gray' 'Cream Tabby/Cream Tabby' 'Black Tabby/Black'\n",
      " 'Cream/Tricolor' 'Yellow/Orange Tabby' 'Orange Tabby/Cream'\n",
      " 'Green/Orange' 'Gray/Silver' 'Tricolor/Brown Brindle' 'Black/Tortie'\n",
      " 'White/Lilac Point' 'Black/Brown Merle' 'Blue Tabby/Tan' 'Fawn/Chocolate'\n",
      " 'Gold/Gold' 'White/Silver' 'Blue/Green' 'Blue Merle/Gray'\n",
      " 'Black Smoke/Black' 'Tan/Red Tick' 'Tan/Brown Brindle' 'Orange Tiger'\n",
      " 'Green/Blue' 'Gray/Gray Tabby' 'Blue Cream/Tortie' 'Blue Merle/Cream'\n",
      " 'Silver Lynx Point/White' 'Brown/Pink' 'Tricolor/Chocolate'\n",
      " 'Red Merle/Tricolor' 'Calico/Blue Cream' 'Red Tick/Red'\n",
      " 'Lilac Point/Cream' 'Tan/Apricot' 'Calico/Brown Tabby' 'Blue Smoke/Brown'\n",
      " 'Brown Tabby/Gray Tabby' 'Brown Brindle/Blue Tick' 'Brown/Red Tick'\n",
      " 'Blue Point/Cream' 'Agouti/Gray' 'Blue Smoke/White' 'Agouti/Brown Tabby'\n",
      " 'Blue/Silver' 'Yellow Brindle/Blue' 'Seal Point/Buff'\n",
      " 'Tortie/Black Smoke' 'Torbie/Black' 'Red Merle/Brown Merle' 'Silver/Gray'\n",
      " 'Green/White' 'Brown Brindle/Blue' 'Black Tiger' 'Black/Brown Tabby'\n",
      " 'Sable/Red' 'White/Black Smoke' 'Lynx Point/Tan' 'Black/Gray Tabby'\n",
      " 'Black Smoke/Brown' 'Chocolate/Brown Merle' 'Red/Green' 'Tricolor/Calico'\n",
      " 'Chocolate/Yellow' 'Black Brindle/Blue Tick' 'Gray/Buff'\n",
      " 'Brown/Blue Merle' 'Brown/Blue' 'Black Brindle/Tan' 'Brown/Black Tabby'\n",
      " 'Brown Merle/Black' 'Cream/Red Tick' 'Blue/Yellow' 'Chocolate/Gray'\n",
      " 'Brown Merle/Chocolate' 'White/Brown Tiger' 'Gray/Fawn'\n",
      " 'Red Merle/Red Merle' 'Tricolor/Orange' 'Yellow/Brown' 'Red Tick/Black'\n",
      " 'Red Tick/Brown Merle' 'Silver/Blue' 'Ruddy/Cream' 'Orange/Blue'\n",
      " 'Lynx Point/Gray' 'Fawn/Gray' 'Blue Merle/Brown Brindle'\n",
      " 'Black Smoke/Chocolate' 'Black Tabby/Gray Tabby' 'Blue Tabby/Orange'\n",
      " 'Brown/Brown Merle' 'Tricolor/Tricolor' 'Chocolate/Red Tick'\n",
      " 'Chocolate/Liver Tick' 'Tortie/Brown' 'Silver Tabby/Black'\n",
      " 'Tan/Cream Tabby' 'Tortie Point/Cream' 'Liver/Liver Tick' 'Cream/Cream'\n",
      " 'Brown Brindle/Brown Merle' 'Tan/Brown Merle' 'Blue/Orange' 'Liver/Buff'\n",
      " 'Brown Tabby/Orange Tabby' 'Tricolor/Brown Merle' 'Lynx Point/Cream'\n",
      " 'Torbie/Blue Cream' 'Blue Smoke/Gray' 'White/Black Tiger'\n",
      " 'Lynx Point/Gray Tabby' 'White/Calico Point' 'Brown Tabby/Black Brindle'\n",
      " 'Tricolor/Red Tick' 'Blue/Yellow Brindle' 'Silver/Cream'\n",
      " 'Brown/Black Brindle' 'Brown Brindle/Blue Cream'\n",
      " 'Cream Tabby/Orange Tabby' 'Brown/Apricot' 'Tortie Point/Blue'\n",
      " 'Blue Cream/Buff' 'Tortie/Blue Tabby' 'Sable/Red Merle'\n",
      " 'Black/Seal Point' 'Agouti/Cream' 'Blue Tabby/Blue Cream' 'White/Green'\n",
      " 'Blue Cream/Blue Tabby' 'Brown Brindle/Gray' 'Torbie/Silver Tabby'\n",
      " 'Red Merle/Tan' 'Buff/Yellow' 'Brown Tabby/Lynx Point' 'Black Tabby/Gray'\n",
      " 'Black/Silver Tabby' 'Chocolate/Brown Brindle' 'Red/Brown Brindle'\n",
      " 'Cream Tiger' 'Orange Tabby/Black' 'Brown Brindle/Liver Tick'\n",
      " 'Blue Tabby/Tortie' 'White/Flame Point' 'Tortie Point/Seal Point']\n"
     ]
    }
   ],
   "source": [
    "for c in text_features:\n",
    "    print(c)\n",
    "    print(df[c].unique()) #value_counts())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We re-use the helper functions from the 'Text processing' notebook above.\n",
    "\n",
    "__Warning__: cleaning stage can take a few minutes, depending on how much text is there to process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Text cleaning:  Name\n",
      "Text cleaning:  Found Location\n",
      "Text cleaning:  Breed\n",
      "Text cleaning:  Color\n"
     ]
    }
   ],
   "source": [
    "# Prepare cleaning functions\n",
    "import re, string\n",
    "import nltk\n",
    "from nltk.stem import SnowballStemmer\n",
    "\n",
    "stop_words = [\"a\", \"an\", \"the\", \"this\", \"that\", \"is\", \"it\", \"to\", \"and\"]\n",
    "\n",
    "stemmer = SnowballStemmer('english')\n",
    "\n",
    "def preProcessText(text):\n",
    "    # lowercase and strip leading/trailing white space\n",
    "    text = text.lower().strip()\n",
    "    \n",
    "    # remove HTML tags\n",
    "    text = re.compile('<.*?>').sub('', text)\n",
    "    \n",
    "    # remove punctuation\n",
    "    text = re.compile('[%s]' % re.escape(string.punctuation)).sub(' ', text)\n",
    "    \n",
    "    # remove extra white space\n",
    "    text = re.sub('\\s+', ' ', text)\n",
    "    \n",
    "    return text\n",
    "\n",
    "def lexiconProcess(text, stop_words, stemmer):\n",
    "    filtered_sentence = []\n",
    "    words = text.split(\" \")\n",
    "    for w in words:\n",
    "        if w not in stop_words:\n",
    "            filtered_sentence.append(stemmer.stem(w))\n",
    "    text = \" \".join(filtered_sentence)\n",
    "    \n",
    "    return text\n",
    "\n",
    "def cleanSentence(text, stop_words, stemmer):\n",
    "    return lexiconProcess(preProcessText(text), stop_words, stemmer)\n",
    "\n",
    "# Clean the text features\n",
    "for c in text_features:\n",
    "    print('Text cleaning: ', c)\n",
    "    df[c] = [cleanSentence(item, stop_words, stemmer) for item in df[c].values]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The cleaned text features are ready to be vectorized after the train/test split."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Note__: more exploratory data analysis might reveal other important hidden atributes and/or relationships of the model features considered. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. <a name=\"4\">Training, validation, and test subsets</a>\n",
    "(<a href=\"#0\">Go to top</a>)\n",
    "\n",
    "We will split our dataset into training (80%), validation (10%), and test (10%) subsets using sklearn's [__train_test_split()__](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html) function twice."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "train_data, test_data = train_test_split(df, test_size=0.2, shuffle=True, random_state=23)\n",
    "val_data, test_data = train_test_split(test_data, test_size=0.5, shuffle=True, random_state=23)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Target balancing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training set shape: (76388, 13)\n",
      "Class 0 samples in the training set: 33357\n",
      "Class 1 samples in the training set: 43031\n",
      "Class 0 samples in the test set: 4163\n",
      "Class 1 samples in the test set: 5386\n"
     ]
    }
   ],
   "source": [
    "print('Training set shape:', train_data.shape)\n",
    "\n",
    "print('Class 0 samples in the training set:', sum(train_data[model_target] == 0))\n",
    "print('Class 1 samples in the training set:', sum(train_data[model_target] == 1))\n",
    "\n",
    "print('Class 0 samples in the test set:', sum(test_data[model_target] == 0))\n",
    "print('Class 1 samples in the test set:', sum(test_data[model_target] == 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Important note:__ We want to fix the imbalance only in training set. We shouldn't change the validation and test sets, as these should follow the original distribution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.utils import shuffle\n",
    "\n",
    "class_0_no = train_data[train_data[model_target] == 0]\n",
    "class_1_no = train_data[train_data[model_target] == 1]\n",
    "\n",
    "upsampled_class_0_no = class_0_no.sample(n=len(class_1_no), replace=True, random_state=42)\n",
    "\n",
    "train_data = pd.concat([class_1_no, upsampled_class_0_no])\n",
    "train_data = shuffle(train_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training set shape: (86062, 13)\n",
      "Class 1 samples in the training set: 43031\n",
      "Class 0 samples in the training set: 43031\n"
     ]
    }
   ],
   "source": [
    "print('Training set shape:', train_data.shape)\n",
    "\n",
    "print('Class 1 samples in the training set:', sum(train_data[model_target] == 1))\n",
    "print('Class 0 samples in the training set:', sum(train_data[model_target] == 0))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. <a name=\"5\">Data processing with Pipeline and ColumnTransformer</a>\n",
    "(<a href=\"#0\">Go to top</a>)\n",
    "\n",
    "We can use the composite Pipeline of Day 2 to train and tune a neural network in sklearn, using its implementation of neural network __MLPClassifier__. However, sklearn is not a neural network framework, lacking access to large scale optimization techniques with GPU support and more neural network related utility functions. \n",
    " \n",
    "We instead build a neural network with __MXNet/Gluon__. While for classic, non-neural algorithms, MXNet/Gluon is not particularly useful, using an actual deep learning framework for neural network experimentation provides more flexibility and customization.\n",
    "\n",
    "Choice of model and hosting platform aside, we can still reuse the collective ColumnTransformer from Day 2 to preprocess the data for neural network training, validation and test, ensuring that the transformations learned on the train data are performed accordingly on the training, validation and test datasets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Datasets shapes before processing:  (86062, 11) (9548, 11) (9549, 11)\n",
      "Datasets shapes after processing:  (86062, 235) (9548, 235) (9549, 235)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.impute import SimpleImputer\n",
    "from sklearn.preprocessing import OneHotEncoder, MinMaxScaler\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.compose import ColumnTransformer\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.metrics import classification_report\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "\n",
    "### COLUMN_TRANSFORMER ###\n",
    "##########################\n",
    "\n",
    "# Preprocess the numerical features\n",
    "numerical_processor = Pipeline([\n",
    "    ('num_imputer', SimpleImputer(strategy='mean')),\n",
    "    ('num_scaler', MinMaxScaler()) # Shown in case is needed, not a must with Decision Trees\n",
    "                                ])\n",
    "                  \n",
    "# Preprocess the categorical features\n",
    "categorical_processor = Pipeline([\n",
    "    ('cat_imputer', SimpleImputer(strategy='constant', fill_value='missing')), # Shown in case is needed, no effect here as we already imputed with 'nan' strings\n",
    "    ('cat_encoder', OneHotEncoder(handle_unknown='ignore')) # handle_unknown tells it to ignore (rather than throw an error for) any value that was not present in the initial training set.\n",
    "                                ])\n",
    "\n",
    "# Preprocess 1st text feature\n",
    "text_processor_0 = Pipeline([\n",
    "    ('text_vect_0', CountVectorizer(binary=True, max_features=50))\n",
    "                                ])\n",
    "\n",
    "# Preprocess 2nd text feature (larger vocabulary)\n",
    "text_precessor_1 = Pipeline([\n",
    "    ('text_vect_1', CountVectorizer(binary=True, max_features=150))\n",
    "                                ])\n",
    "\n",
    "# Combine all data preprocessors from above (add more, if you choose to define more!)\n",
    "# For each processor/step specify: a name, the actual process, and finally the features to be processed\n",
    "data_preprocessor = ColumnTransformer([\n",
    "    ('numerical_pre', numerical_processor, numerical_features),\n",
    "    ('categorical_pre', categorical_processor, categorical_features),\n",
    "    ('text_pre_0', text_processor_0, text_features[0]),\n",
    "    ('text_pre_1', text_precessor_1, text_features[1])\n",
    "                                    ]) \n",
    "\n",
    "### DATA PREPROCESSING ###\n",
    "##########################\n",
    "\n",
    "# Get train data to train the network\n",
    "X_train = train_data[model_features]\n",
    "y_train = train_data[model_target]\n",
    "\n",
    "# Get validation data to validate the network \n",
    "X_val = val_data[model_features]\n",
    "y_val = val_data[model_target]\n",
    "\n",
    "# Get test data to test the network\n",
    "X_test = test_data[model_features]\n",
    "y_test = test_data[model_target]\n",
    "\n",
    "print('Datasets shapes before processing: ', X_train.shape, X_val.shape, X_test.shape)\n",
    "\n",
    "X_train = data_preprocessor.fit_transform(X_train).toarray()\n",
    "X_val = data_preprocessor.transform(X_val).toarray()\n",
    "X_test = data_preprocessor.transform(X_test).toarray()\n",
    "\n",
    "print('Datasets shapes after processing: ', X_train.shape, X_val.shape, X_test.shape)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. <a name=\"6\">Neural network training and validation</a>\n",
    "(<a href=\"#0\">Go to top</a>)\n",
    "\n",
    "We will use PyTorch to build a simple neural network and fit to our training data. We will also use our validation data and check performance at the end of each iteration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1, training loss: 0.51, validation loss: 0.40, training accuracy: 0.76, validation accuracy: 0.83\n",
      "Epoch 2, training loss: 0.41, validation loss: 0.39, training accuracy: 0.82, validation accuracy: 0.84\n",
      "Epoch 3, training loss: 0.40, validation loss: 0.38, training accuracy: 0.82, validation accuracy: 0.85\n",
      "Epoch 4, training loss: 0.40, validation loss: 0.38, training accuracy: 0.83, validation accuracy: 0.84\n",
      "Epoch 5, training loss: 0.40, validation loss: 0.38, training accuracy: 0.83, validation accuracy: 0.84\n",
      "Epoch 6, training loss: 0.39, validation loss: 0.37, training accuracy: 0.83, validation accuracy: 0.84\n",
      "Epoch 7, training loss: 0.39, validation loss: 0.38, training accuracy: 0.83, validation accuracy: 0.84\n",
      "Epoch 8, training loss: 0.39, validation loss: 0.37, training accuracy: 0.83, validation accuracy: 0.85\n",
      "Epoch 9, training loss: 0.39, validation loss: 0.37, training accuracy: 0.83, validation accuracy: 0.85\n",
      "Epoch 10, training loss: 0.39, validation loss: 0.37, training accuracy: 0.83, validation accuracy: 0.85\n",
      "Epoch 11, training loss: 0.38, validation loss: 0.37, training accuracy: 0.83, validation accuracy: 0.85\n",
      "Epoch 12, training loss: 0.38, validation loss: 0.37, training accuracy: 0.83, validation accuracy: 0.85\n",
      "Epoch 13, training loss: 0.38, validation loss: 0.37, training accuracy: 0.83, validation accuracy: 0.85\n",
      "Epoch 14, training loss: 0.38, validation loss: 0.36, training accuracy: 0.83, validation accuracy: 0.85\n",
      "Epoch 15, training loss: 0.38, validation loss: 0.37, training accuracy: 0.83, validation accuracy: 0.85\n",
      "Classification report \n",
      "               precision    recall  f1-score   support\n",
      "\n",
      "         0.0       0.87      0.76      0.81      4111\n",
      "         1.0       0.83      0.91      0.87      5437\n",
      "\n",
      "    accuracy                           0.85      9548\n",
      "   macro avg       0.85      0.84      0.84      9548\n",
      "weighted avg       0.85      0.85      0.84      9548\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.optim as optim\n",
    "from torch.utils.data import TensorDataset, DataLoader\n",
    "import numpy as np\n",
    "from sklearn.metrics import confusion_matrix, accuracy_score, classification_report\n",
    "\n",
    "# Set this to CPU or GPU depending on your training instance\n",
    "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "\n",
    "# Set data loaders\n",
    "X_train_tensor = torch.FloatTensor(X_train)\n",
    "y_train_tensor = torch.FloatTensor(y_train.values)\n",
    "X_val_tensor = torch.FloatTensor(X_val)\n",
    "y_val_tensor = torch.FloatTensor(y_val.values)\n",
    "\n",
    "train_dataset = TensorDataset(X_train_tensor, y_train_tensor)\n",
    "val_dataset = TensorDataset(X_val_tensor, y_val_tensor)\n",
    "\n",
    "train_loader = DataLoader(train_dataset, batch_size=16, shuffle=True)\n",
    "val_loader = DataLoader(val_dataset, batch_size=16)\n",
    "\n",
    "# Create a simple MultiLayer Perceptron\n",
    "class Net(nn.Module):\n",
    "    def __init__(self, input_size):\n",
    "        super(Net, self).__init__()\n",
    "        self.fc1 = nn.Linear(input_size, 64)\n",
    "        self.fc2 = nn.Linear(64, 64)\n",
    "        self.fc3 = nn.Linear(64, 1)\n",
    "        self.relu = nn.ReLU()\n",
    "        self.dropout = nn.Dropout(0.4)\n",
    "        self.sigmoid = nn.Sigmoid()\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.relu(self.fc1(x))\n",
    "        x = self.dropout(x)\n",
    "        x = self.relu(self.fc2(x))\n",
    "        x = self.dropout(x)\n",
    "        x = self.sigmoid(self.fc3(x))\n",
    "        return x\n",
    "\n",
    "net = Net(X_train.shape[1]).to(device)\n",
    "\n",
    "# Hyper-parameters of the system\n",
    "epochs = 15\n",
    "learning_rate = 0.01\n",
    "\n",
    "# Define loss and optimizer\n",
    "criterion = nn.BCELoss()\n",
    "optimizer = optim.SGD(net.parameters(), lr=learning_rate)\n",
    "\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Training loop\n",
    "for epoch in range(epochs):\n",
    "    net.train()\n",
    "    train_loss = 0.0\n",
    "    train_predictions = []\n",
    "    train_labels = []\n",
    "\n",
    "    for data, labels in train_loader:\n",
    "        data, labels = data.to(device), labels.to(device)\n",
    "        optimizer.zero_grad()\n",
    "        outputs = net(data)\n",
    "        loss = criterion(outputs, labels.unsqueeze(1))\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "        train_loss += loss.item() * data.size(0)\n",
    "        train_predictions.extend(outputs.detach().cpu().numpy())\n",
    "        train_labels.extend(labels.cpu().numpy())\n",
    "\n",
    "    train_loss = train_loss / len(train_loader.dataset)\n",
    "    train_predictions = (np.array(train_predictions) >= 0.5).astype(int)\n",
    "    train_accuracy = accuracy_score(train_labels, train_predictions)\n",
    "\n",
    "    # Validation\n",
    "    net.eval()\n",
    "    val_loss = 0.0\n",
    "    val_predictions = []\n",
    "    val_labels = []\n",
    "\n",
    "    with torch.no_grad():\n",
    "        for data, labels in val_loader:\n",
    "            data, labels = data.to(device), labels.to(device)\n",
    "            outputs = net(data)\n",
    "            loss = criterion(outputs, labels.unsqueeze(1))\n",
    "            val_loss += loss.item() * data.size(0)\n",
    "            val_predictions.extend(outputs.cpu().numpy())\n",
    "            val_labels.extend(labels.cpu().numpy())\n",
    "\n",
    "    val_loss = val_loss / len(val_loader.dataset)\n",
    "    val_predictions = (np.array(val_predictions) >= 0.5).astype(int)\n",
    "    val_accuracy = accuracy_score(val_labels, val_predictions)\n",
    "    val_report = classification_report(val_labels, val_predictions)\n",
    "\n",
    "    print(f\"Epoch {epoch+1}, training loss: {train_loss:.2f}, validation loss: {val_loss:.2f}, \"\n",
    "          f\"training accuracy: {train_accuracy:.2f}, validation accuracy: {val_accuracy:.2f}\")\n",
    "print(\"Classification report \\n\", val_report)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7. <a name=\"7\">Test the neural network</a>\n",
    "(<a href=\"#0\">Go to top</a>)\n",
    "\n",
    "Let's see how the network performs with \"unseen\" data (our test data)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[32814 10217]\n",
      " [ 3572 39459]]\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "         0.0       0.90      0.76      0.83     43031\n",
      "         1.0       0.79      0.92      0.85     43031\n",
      "\n",
      "    accuracy                           0.84     86062\n",
      "   macro avg       0.85      0.84      0.84     86062\n",
      "weighted avg       0.85      0.84      0.84     86062\n",
      "\n",
      "Train accuracy: 0.8397782993655737\n",
      "[[3154 1009]\n",
      " [ 450 4936]]\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "         0.0       0.88      0.76      0.81      4163\n",
      "         1.0       0.83      0.92      0.87      5386\n",
      "\n",
      "    accuracy                           0.85      9549\n",
      "   macro avg       0.85      0.84      0.84      9549\n",
      "weighted avg       0.85      0.85      0.85      9549\n",
      "\n",
      "Test accuracy: 0.8472091318462667\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import confusion_matrix, classification_report, accuracy_score\n",
    "\n",
    "net.eval()\n",
    "\n",
    "# Predictions on train data\n",
    "train_input = torch.FloatTensor(X_train).to(device)\n",
    "with torch.no_grad():\n",
    "    train_predictions = net(train_input).cpu().numpy()\n",
    "train_predictions = (train_predictions >= 0.5).astype(int).flatten()\n",
    "\n",
    "print(confusion_matrix(y_train, train_predictions))\n",
    "print(classification_report(y_train, train_predictions))\n",
    "print(\"Train accuracy:\", accuracy_score(y_train, train_predictions))\n",
    "\n",
    "# Predictions on test data\n",
    "test_input = torch.FloatTensor(X_test).to(device)\n",
    "with torch.no_grad():\n",
    "    test_predictions = net(test_input).cpu().numpy()\n",
    "test_predictions = (test_predictions >= 0.5).astype(int).flatten()\n",
    "\n",
    "print(confusion_matrix(y_test, test_predictions))\n",
    "print(classification_report(y_test, test_predictions))\n",
    "print(\"Test accuracy:\", accuracy_score(y_test, test_predictions))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 8. <a name=\"8\">Improvement Ideas</a>\n",
    "(<a href=\"#0\">Go to top</a>)\n",
    "\n",
    "* Further tune network parameters (architecture, # layers, # hidden neurons, activation functions, weights initialization, dropout, optimizer, learning rate, batch size, # epochs), while closely monitoring the loss function and the accuracy on both training and validation as a function of number of epochs.\n",
    "* Experiment with different PyTorch optimizers like Adam or RMSprop.\n",
    "* Implement learning rate scheduling to improve convergence.\n",
    "* Use PyTorch's built-in features for early stopping and model checkpointing.\n",
    "* Explore more advanced PyTorch modules like nn.Sequential for cleaner network definition."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "sagemaker-distribution:Python",
   "language": "python",
   "name": "conda-env-sagemaker-distribution-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
